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Biological  and  Silicon  Modeling  of  Moving  Target 

Detection  in  Insects 

Executive  Summary 

In  this  project,  we  studied  the  physiology  of  a  class  of  visual  neurons  that  we  have 
labeled  small  target,  movement  detectors  (STMDs),  found  in  the  third  optic  ganglion  of 
insect  species  of  several  orders,  which  are  notable  for  the  aerial  pursuit  of  prey  or 
conspecifics.  These  neurons  have  selective  responses  for  small  moving  targets  in  various 
parts  of  the  animals’  visual  fields.  In  addition,  we  developed  models  for  aspects  of  the 
processing  performed  by  these  cells,  and  designed  and  fabricated  analog  integrated 
circuits  based  on  these  models. 

In  the  neurobiological  portion  of  our  work,  the  primary  experimental  tool  we  used 
was  in-vivo  intracellular  recording  in  animals  of  the  model  species  (the  hoverfly  Eristalis 
tenax  and  the  dragonflies  Hemicordulia  tau  and  Aeshna  multicolor),  while  subject  to 
moving  visual  displays.  We  characterized  the  responses  of  a  range  of  neurons  that  can  be 
considered  small  target  movement  detectors;  developed  preliminary  classifications  for 
some  of  these  cells,  including  a  small-field  STMD  that  we  have  labeled  an  ‘elementary 
small  target  movement  detector’,  or  ESTMD,  and  which  may  be  an  early  or  intermediate 
stage  in  a  hierarchy  of  STMD  processing;  discovered  that  at  least  some  STMD  neurons 
are  capable  of  responding  selectively  to  small  moving  targets  against  moving  cluttered 
backgrounds;  and  obtained  data  from  an  STMD  neuron  that  we  believe  will  prove 
identifiable  from  individual  to  individual  of  a  particular  species  {E.  tenax). 

In  our  modeling  work,  we  developed  both  conceptual  and  numerical  models  for 
aspects  of  STMD  processing.  Simulations  were  performed  using  the  tools  MatLab  and 
SPICE,  in  which  time-domain  stimuli  (in  many  cases  the  same  or  similar  to  that 
presented  to  insects  during  experiments)  were  used  as  inputs.  We  modeled  the  processing 
performed  by  ESTMDs;  studied  the  responses  of  elementary  motion  detectors  (the 
putative  front-end  processors  for  subsequent  for  STMD  neurons)  to  small  target  motion; 
developed  a  model  for  processing  that  could  endow  STMD  neurons  with  selectivity  for 
small  target  size  in  the  direction  of  motion;  examined  possible  mechanisms  for  collation 
of  ESTMD  outputs;  and  developed  a  revised  hierarchy  of  STMD  processing  in  view  of 
the  discovery  of  neiuons  capable  of  distinguishing  moving  targets  against  moving 
backgrounds. 

In  our  effort  to  implement  neuromorphic  analog  circuit  models  of  STMDs,  we 
evaluated  primitive  devices  and  STMD  models  in  two  different  CMOS  processes 
(standard  bulk  silicon  mixed-signal  CMOS  and  silicon-on-sapphire);  invented  a  novel 
log-domain  filter  circuit  that  can  be  operated  in  a  nonlinear  mode  to  mimic  the  ‘tonic’  or 
persistent  response  property  of  ESTMD  neurons;  implemented,  tested  and  demonstrated 
silicon  models  for  the  ESTMD;  and  implemented  and  performed  pilot  testing  of  a  model 
for  dendritic  collation  in  a  wide-field  STMD  analog. 
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1  INTRODUCTION 

1.1  Background 

1.2  Objectives 

The  stated  objectives  of  the  proposed  work  were: 

1.  Development  of  a  biological  model  of  insect  small  target,  movement  detector 
neurons  (STMDs)  to  a  sufficiently  quantitative  and  abstract  state  to  allow  an 
artificial  system  to  be  based  upon  its  principles  of  operation; 

2.  Demonstration  of  the  feasibility  of  an  artificial  STMD  implementation,  in  the 
form  of  an  analog  silicon  model. 

Although  the  work  came  to  focus  on  particular  classes  of  STMD  cells  and  aspects  of 
their  behavior,  the  two  stated  objectives  of  the  original  proposal  remained  fimdamentally 
unchanged.  However,  as  the  work  progressed,  the  range  of  STMD  neurons  found  made 
evident  the  need  for  fiulher  basic  characterization  of  these  cells  and  their  physiologcal 
characteristics,  and  this  became  an  additional  de  facto  objective  of  the  experimental  part 
of  the  effort. 

1.3  Approach 

Our  work  has  involved  experimental  neurobiology,  numerical  modeling  and 
simulation,  and  development  of  analog  silicon  models  of  STMD  functions.  The  primary 
experimental  tool  has  been  in-vivo  intracellular  recording  from  putative  STMD  neurons 
in  intact  animals  (dragonflies  and  hoverflies)  while  subject  to  stimulation  via  a  visual 
display.  In  parallel  with  experimental  neurophysiology,  modeling  and  simulation  was 
conducted  at  several  levels:  both  biologically  detailed  and  higher-level,  abstract  models 
were  developed.  This  paradigm  generates  testable  hypotheses  for  experimental 
verification,  and  promotes  the  abstraction  of  essential  computational  principles  that 
underlie  physiological  responses.  Silicon  models  have  built  upon  the  results  of 
simulation-based  modeling.  STMD  models  were  fabricated  in  analog  integrated  circuitry 
using  submicron,  mixed-signal  CMOS  processes,  and  neuromorphic  design  techniques. 


2  METHODS  AND  PROCEDURES 

2.1  Experimental  Neurobiology 

The  major  experimental  tool  for  this  project  has  been  in-vivo  intracellular  recording 
in  intact  animals  while  they  are  subject  to  stimulation  via  a  visual  display.  The  species 
used  in  the  preparations  have  been  the  dragonflies  Aeshna  multicolor  and  Hemicordulia 
tau  (order  Odonata),  and  the  hoverfly  Eristalis  tenax  (order  Diptera),  all  animals  that 
locate  and  pursue  prey  or  conspecifics  visually.  During  the  experimental  procedure,  the 
animals  are  immobilized,  a  small  portion  of  the  rear  head  capsule  is  removed,  and  a 
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drawn  quartz  or  aluminosilicate  glass  microelectrode  filled  with  electrolyte  solution,  with 
a  tip  of  less  than  lOOnm  diameter,  is  inserted  to  penetrate  individual  neurons.  A  reference 
electrode  (fine  gauge  silver  chloride  coated  silver  wire),  is  inserted  into  the  rear  of  the 
eye  contralateral  to  that  from  which  recordings  are  obtained.  The  potential  difference 
between  this  reference  and  the  internal  membrane  potential  of  the  recorded  cell  is 
digitized  as  moving  visual  scenarios  are  displayed  to  the  animal  on  a  CRT  screen. 

Data  recorded  during  such  experiments  were  analyzed,  and  electrophysiological 
events  such  as  changes  in  graded  potential  or  rate  of  firing  of  action  potentials  were 
correlated  with  events  (in  particular,  small  target  motion)  in  the  stimulus  sequence.  As 
the  project  progressed  and  hypotheses  regarding  the  processing  performed  by  STMD 
neurons  were  generated,  specialized  stimulus  sequences  were  developed  for  further 
experiments. 

Considerable  effort  was  expended  during  the  course  of  the  project  on  two  major 
technical  issues  relating  to  the  experimental  preparation  and  procedures:  one  had  to  do 
with  improving  the  stability  and  quality  of  the  electrophysiological  data,  and  the  second 
with  advancing  stimulus  display  capabilities. 

Successful  recording  from  STMD  neurons  is  more  difficult  than  from  the  large,  wide- 
field  tangential  cells  in  the  lobula  plate  which  have  been  a  staple  of  insect  vision  research 
in  ours  as  well  as  other  laboratories.  Our  work  with  STMD  cells  has  spurred  the 
development  of  specialized  techniques  during  the  course  of  the  project.  Initial  recording 
attempts  were  hampered  by  signal  degradation  due  to  electrical  noise  intruding  into 
recordings.  Much  of  this  noise  emanates  from  the  computer  display  on  which  stimuli  are 
presented,  which,  given  its  proximity  to  the  recording  site,  presents  a  substantial 
challenge  to  shielding.  In  recordings  from  tangential  cells,  noise  has  not  been  a  major 
issue,  since  we  are  able  to  low-pass  filter  the  electrical  response  from  these  cells  (which 
are  non-spiking),  and  to  use  relatively  low-impedance  recording  electrodes  (ca.  50-70 
MQ).  STMD  neurons  are  generally  much  smaller  than  tangential  cells.  Preliminary 
analysis  of  recorded  cells  after  dye  injection  suggests  that  our  recordings  in  vivo  are  often 
from  neural  processes  below  200nm  diameter,  necessitating  the  use  of  extremely  fine 
microelectrodes.  Our  initial  recording  attempts  from  STMDs  with  the  low  impedance 
electrodes  were  partially  successful,  in  that  we  were  able  to  penetrate  neurons  that 
responded  to  small  targets,  but  we  ‘lost’  cells  after  a  short  recording  period  (<I  minute), 
possibly  due  to  damage  caused  by  the  electrode  tip  as  it  first  penetrates  the  cell.  We 
subsequently  developed  modified  programs  on  our  Sutter  P-97  electrode  puller  to 
fabricate  finer-tipped  electrodes,  which  give  more  stable  recordings  from  STMD  cells 
(although  the  recording  duration  is  still  typically  minutes  in  duration  rather  than  the  hours 
that  can  be  achieved  with  tangential  cells).  Unfortimately,  the  higher  impedance  (120- 
180MQ)  of  these  electrodes  is  poorly  matched  to  the  preamplifier  head  stage  and  results 
in  additional  noise  pickup. 

This  problem  is  exacerbated  by  the  fact  that  most  STMD  neurons  fire  ‘fast’  action 
potentials  (<1  ms  duration  for  the  positive  phase),  which  require  a  much  higher  sample 
rate  (5  kHz)  to  digitize  reliably  and  which  therefore  preclude  the  same  degree  of  low-pass 
filtering  that  is  acceptable  for  tangential  cell  recordings.  Since  the  noise  from  the  display 
is  at  relatively  high  frequencies,  this  leads  to  worsening  of  the  recording  signal-to-noise 
ratio.  We  have  tackled  this  problem  by  careful  attention  to  electrical  shielding,  including 
the  use  of  a  grounded  transparent  thin-film  metallic  acetate  material  (Tecknit)  material 


Contract  F49620-01-C-0030  Final  Report 


5 


that  reduces  noise  emanation  through  the  CRT.  One  tradeoff  with  this  approach  is  that 
this  screen  reduces  the  mean  luminance  of  the  display  by  a  factor  of  two.  This  is  of 
concern  since  the  display  luminance  is  already  much  lower  than  that  of  the  typical  sky 
against  which  targets  would  be  viewed  in  nature.  While  a  brighter  display  would  have 
been  desirable,  we  were  not  successful  in  locating  a  suitable  high-speed  bright-phosphor 
CRT  display  during  the  course  of  the  project.  Any  CRT  used  in  these  experiments  must 
be  capable  of  a  200  frame  per  second  vertical  refresh  (frame  rate)  in  order  to  exceed  the 
flicker  fusion  frequency  in  the  temporally  acute  insect  eye. 

During  the  first  half  of  the  project,  we  were  also  best  by  an  intermittent  low- 
frequency  noise  problem,  eventually  traced  to  a  break-down  of  the  Ag-AgCl  coating 
placed  on  our  reference  (ground)  electrodes.  This  layer  is  formed  on  silver  wire  by 
electroplating.  Upon  fiirther  reading  on  this  subject  we  found  that  AgCl  layer  instability 
could  have  resulted  from  the  relatively  high  voltage  (9V)  we  were  using  to  coat  the  silver 
wire:  we  found  that  reducing  the  potential  during  coating  to  1.5  Volts  produces  a  more 
stable  AgCl  layer  and  this  led  to  a  complete  elimination  of  the  intermittent  low-frequency 
noise. 

Generation  of  display  stimuli  for  the  experiments  was  also  a  significant  technical 
challenge.  All  of  our  preliminary  neurobiological  data  were  obtained  using  relatively 
dated  stimulus  display  techniques,  including  a  Picasso  image  synthesizer  and  a 
Cambridge  Research  Systems  VSG  display  card.  These  proprietary  systems  are 
expensive  and  have  many  limitations  for  particular  applications,  but  are  necessary  to 
produce  the  very  high  frame  rates  required  for  simulation  of  moving  scenes  and  patterns 
when  working  with  insects.  However,  during  the  course  of  the  project,  Andrew  Straw,  a 
graduate  student  in  the  O'Carroll  laboratory,  developed  a  new  display  system,  the  "Vision 
Egg,"  under  partial  support  of  this  contract. 

Computer  graphics  capabilities  have  been  driven  in  recent  years  by  the  requirements 
of  commercial  computer  games,  and  the  latest  generation  AGP-based  graphics  cards  that 
are  now  standard  in  modem  computers  have  capabilities  that  exceed  those  of  the 
traditional  display  systems  used  by  vision  scientists.  They  are  capable  of  drawing  very 
complex  scenes  at  high  frame  rates.  The  Vision  Egg  is  a  programming  library  that  uses 
these  standard,  inexpensive  computer  graphics  cards  to  produce  visual  stimuli  for  vision 
research  experiments.  It  allows  programming  of  the  cards  with  OpenGL,  an  open-source 
standard  for  computer  graphics  programming.  Potentially  difficult  tasks,  such  as 
initializing  graphics,  getting  precise  timing  information,  controlling  stimulus  parameters 
in  real-time,  and  synchronizing  with  data  acquisition,  are  expedited  by  routines  within  the 
Vision  Egg  library.  The  Vision  Egg  is  an  open-source  project  licensed  under  the  GNU 
GPL.  To  facilitate  distribution  to  the  vision  science  community,  we  have  published  it  on 
an  open  source  web  site,  http://www.visionegg.org.  Evaluation  copies  of  the  code  can  be 
freely  downloaded  by  other  vision  scientists. 

Using  the  Vision  Egg,  we  have  developed  a  progressively  more  sophisticated 
repertoire  of  stimulus  capabilities  during  the  course  of  the  project,  progressing  from  a 
small  target  drifting  unidirectionally  against  a  blank  screen,  to  a  small  target  moving 
along  arbitrary  paths  (such  as  orbits)  against  cluttered  backgrounds  (including  natural 
imagery),  to  small  targets  moving  against  drifting  cluttered  backgrounds  with  interactive 
selection  of  target  location  in  the  scene. 
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Finally,  we  have  improved  data  acquisition  techniques  and  hardware,  by  writing  data 
acquisition  code  in  Matlab,  procuring  a  new  (National  Instruments)  data  acquisition  card 
that  offers  substantial  noise  advantages  over  the  prior  GWI-625  card  used  in  our 
laboratory,  and  integrating  these  with  the  Vision  Egg  for  purposes  of  sychronization  of 
display  with  data  acquisition. 

2.2  Modeling  and  Simulation 

Conceptual  models  for  the  processing  performed  by  particular  STMD  neurons  were 
developed  and  refined  as  experimental  work  proceeded  during  the  project.  As  these 
models  became  sufficiently  well-defined  that  they  could  be  expressed  mathematically, 
they  were  coded  in  software,  and  simulations  were  performed  to  verify  their  response 
characteristics  and  investigate  their  behaviors.  This  process  eventually  generated  testable 
hypotheses  which  in  some  instances  could  be  used  as  feedback  for  the  neurobiology  task, 
to  design  experiments  for  more  detailed  investigation  of  STMD  physiology. 

Numerical  modeling  and  simulation  were  performed  in  several  environments.  In 
general,  the  various  simulation  tools  were  used  to  eompute  transient  time-domain 
responses  of  dynamic  systems  corresponding  to  the  STMD  models,  to  particular 
simulated  stimuli  (in  some  cases,  the  same  stimuli  that  were  presented  to  subject  animals 
during  experiments).  The  academic  investigator  conducted  simulations  in  which  the 
models  maintained  a  relatively  high  degree  of  fidelity  to  the  biological  system,  primarily 
using  the  mathematical  package  Matlab  with  custom-written  code.  Modeling  of  a  more 
abstract  nature  was  carried  out  by  the  commercial  partner  using  the  tool  SPICE 
(Simulation  Program  with  Integrated  Circuit  Emphasis).  SPICE  is  based  on  circuit 
representations,  but  admits  the  use  of  purely  abstract  components  in  addition  to  models  of 
particular  electronic  devices.  In  SPICE  simulations,  we  attempted  to  verify  what  we 
believed  to  be  the  essential  computational  features  of  STMD  processing  seen  and 
modeled  in  the  biological  neurons.  Finally,  SPICE  also  served  as  a  platform  for  device- 
based  circuit  models  during  the  process  of  circuit  design.  Simulations  of  the  circuits 
actually  implemented  in  the  silicon  modeling  task  were  performed  (or  at  least  attempted) 
in  order  to  verify  their  response  characteristics. 

It  is  presumed  that  considerable  early  visual  and  motion  processing  takes  place  in  the 
visual  pathway  prior  to  the  STMD  neurons,  and  some  simulations  (with  biological 
models  in  particular)  attempted  to  aceount  for  this  processing  by  including  it  in  the 
modeling  in  order  to  form  realistic  input  signals  for  the  STMDs  themselves.  We  have 
assumed  as  a  starting  hypothesis  that  elementary  motion  detectors  (EMDs)  form  the 
principal  inputs  for  STMD  processing,  and  in  fact,  a  substantial  effort  was  devoted  to 
simulating  the  responses  of  the  EMDs  themselves  to  the  passage  of  small  moving  objects, 
in  order  to  understand  the  characteristics  of  their  responses  that  might  be  particular  to  the 
scenario  of  interest. 

The  processing  leading  up  to  the  EMD  output,  as  implemented  in  our  simulations,  can 
be  summarized  as  follows.  The  compound  eye  optics,  in  accordance  with  prior  studies, 
were  assumed  to  have  a  blurring  or  spatial  lowpass  characteristic  with  a  spatial  constant 
or  half-width  on  the  order  of  the  inter-ommatidial  separation.  Photoreceptors  were 
assumed  to  perform  a  roughly  logarithmic  transformation  on  the  received  light  intensity, 
and  to  have  a  predominantly  lowpass  temporal  characteristic.  This  logarithmic 
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dependence  was  in  some  cases  replaced  with  a  more  detailed  model  such  as  the  Naka- 
Rushton  or  Lipetz  model  [4]  [6].  In  the  lamina,  further  temporal  filtering  with  a 
predominantly  bandpass  characteristic  was  assumed  to  occur.  Next,  the  EMD  was 
represented  with  the  correlational  or  Reichardt  model  [1][2].  Frequently,  this  model  was 
elaborated  with  biologically  realistic  features,  such  as  saturating  nonlinearities  in  the 
signal  path.  Finally,  in  accordance  with  the  work  of  O’Carroll  et  al.  [1][7],  contrast  gain 
control  was  in  some  instances  implemented  in  conjimction  with  the  EMD. 

In  other  simulations,  additional  preprocessing  of  some  kind  not  explicitly  simulated 
was  assumed  to  take  place,  and  the  inputs  to  a  particular  stage  of  STMD  processing  were 
assigned  certain  characteristics.  For  instance,  we  assumed  in  some  simulations  that 
signals  from  EMDs  had  been  prefiltered  for  events  consistent  with  the  passage  of  a  small 
moving  target,  and  that  this  prefiltering  resulted  in  an  excitatory  burst  or  pulse  of  a 
constant  amplitude  when  such  an  event  occurred. 

Preliminary  processing  from  early  vision  through  the  EMD,  and  the  first  stages  of 
small  target  detection  as  well,  were  modeled  as  taking  place  on  retinotopic  arrays  of 
processors.  Initially,  simulations  were  performed  with  small,  rectangular  arrays,  and  with 
imiaxial  motion.  However,  before  the  end  of  the  project,  we  developed  tools  to  resample 
moving  imagery  from  standard  video  formats  onto  hexagonal  arrays  analogous  to  the 
actual  geometry  of  photoreceptors  in  the  compound  eye,  and  to  perform  simulations  with 
two-dimensional  motion  using  such  arrays.  (Work  by  O’Carroll  and  Shoemaker  in  this 
area  received  substantial  support  from  a  concurrent  SBIR  project  (Navy  contract 
N00014-03-M-0171),  while  work  by  DuBois  in  the  O’Carroll  lab  was  supported  by  this 
contract.) 

2.3  Circuit  Design,  Implementation,  and  Testing 

Based  on  models  developed  for  the  processing  performed  by  STMD  cells,  we 
developed  experimental  analog  circuits  to  implement  functions  observed  in  biological 
STMD  neurons.  In  the  spirit  of  Mead  and  coworkers  [5],  we  focused  on  micropower  or 
subthreshold  MOS  devices,  relying  primarily  on  the  computational  primitives  available 
in  that  regime.  In  addition  to  physical  modeling  of  functions  associated  with  moving 
target  detection,  we  considered  limitations  of  microelectronic  technology  (in  particular, 
matching  of  device  characteristics  from  transistor  to  transistor)  that  have  the  potential  to 
negatively  impact  robust,  large-scale  practical  implementations  in  the  future. 

For  flexibility,  we  chose  in  the  early  portion  of  the  project  to  evaluate  multiple 
submicron  CMOS  processes  as  potential  media  for  implementation  of  our  models.  The 
processes  considered  were  accessed  via  the  MOSIS  prototyping  service.  Initially,  we 
selected  a  standard  mixed-signal  bulk  CMOS  process  (TSMC  O.SSpm),  and  a  silicon-on- 
sapphire  (SOS)  process  (Peregrine  Semiconductor  O.Spm)  in  which  devices  are 
fabricated  in  a  thin  silicon  film  on  insulating  substrate.  The  SOS  process  results  in 
dielectric  isolation  of  transistors,  and  allows  fabrication  of  a  number  of  primitive  devices 
unavailable  in  bulk  silicon.  Prior  to  submission  of  circuits  for  fabrication,  however,  we 
elected  to  change  vendors  for  bulk  process  and  use  the  Agilent  (formerly  HP)  0.5pm 
mixed-signal  CMOS  process.  This  change  was  due  to  receipt  of  information  suggesting 
superiority  of  device  matching  characteristics  in  the  Agilent  process. 
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Our  approach  consisted  of  the  design,  submission,  and  test  and  evaluation  of  a  series 
of  test  chips.  These  were  intended  to  allow  incremental  development  of  devices  and 
circuit  concepts  for  STMD  models.  The  chip  design  process  comprises  conceptual 
design,  schematic  entry,  circuit  simulation  for  more  complex  circuits,  physical  design 
(layout  of  mask  data),  verification  (design  rule  checking  of  the  layout  and  comparison 
with  the  schematics),  and  transport  of  the  layout  data  to  the  vendor  (MOSIS). 
Simulations  of  higher  level  functions  associated  with  implemented  STMD  models  are 
discussed  in  the  sections  in  Modeling  and  Simulation  (2.2  and  3.2). 

2.3.1  Early  Test  Devices 

Early  in  the  project,  an  unplanned  opportunity  arose  to  place  some  devices  and 
circuits  on  an  SOS  chip  fabricated  under  other  funding,  and  to  evaluate  some  existing 
devices  on  yet  another  SOS  chip.  We  focused  on  evaluating  primitive  devices  that  were 
considered  to  be  of  potential  use  in  future  silicon  models  of  STMDs.  As  mentioned 
above,  a  number  of  such  devices  are  not  feasible  in  an  ordinary  bulk  process,  and  because 
these  were  typically  non-standard  even  in  SOS,  we  developed  technology  files  for  our 
design  environment  allowing  representation  and  extraction  of  all  novel  devices.  These 
files  include  a  library  of  schematic  symbols  for  entry  of  schematic  circuit  diagrams,  and  a 
layout  file  with  layer  definitions  allowing  the  physical  design  of  the  novel  devices  and 
extraction  for  verification. 

Ultimately,  the  primitive  devices  placed  on  the  gratis  SOS  test  chip  included: 

■  MOSFETs  with  body  connections  that  are  also  operable  as  gated  lateral  bipolar 
junction  transistors  (BJTs),  in  both  edged  and  edgeless  configurations; 

■  Ungated  lateral  BJTs; 

■  Depletion-mode  MOSFETs  (useful  for  self-biasing  above-threshold  circuits  and 
moderately  high  resistors); 

■  Depletion  capacitors  for  implementation  of  relatively  linear,  high-value  capacitances. 

Some  simple  multi-transistor  circuits  were  also  placed  on  the  chip,  including  lossless 
BJT  current  mirrors  with  MOSFET  base  drivers,  and  a  simple  log-domain  low-pass 
current  filter  that  was  expected  to  allow  implementation  of  very  long  time  constants.  The 
gratis  chip  was  submitted  in  early  October  2001. 

2.3.2  Chip  1 

Subsequently,  we  designed  a  pair  of  test  chips  for  submission  to  both  the  bulk  CMOS 
and  SOS  processes.  (These  first  two  chips  contained  some  circuits  in  common  with  a 
closely-related,  Air-Force-funded  project  (SBIR  contract  F08630-02-C-0013),  and 
fabrication  costs  were  shared  between  the  two  projects.) 

The  silicon-on-sapphire  test  chip,  chip_l_sos,  was  submitted  to  a  MOSIS  /  Peregrine 
Semiconductor  fabrication  run  closing  6  May  2002.  The  primary  circuit  on  this  chip  was 
an  array  of  ‘elementary  small  target  motion  detectors,’  or  ESTMDs,  a  concept  based  on 
experimental  results  and  analysis  firom  a  class  of  small-field  STMD  neurons  studied 
experimentally  during  the  project  (see  Sections  3.1.3  and  3.2  below).  Addressing 
circuitry  for  purposes  of  handling  inputs  to  this  array  was  also  integrated  with  the  circuit. 
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The  chip  also  contained  a  number  of  subcircuits  and  devices  of  interest,  including  test 
structures  to  evaluate: 

■  MOS  transistor  matching  (devices  with  regular  and  light  channel  dopings) 

■  MOS  transistor  leakage  currents,  in  edged  and  edgeless  devices  (this  limits  the  time 
constants  attainable  in  dynamic  circuits) 

■  Lateral  bipolar  junction  transistor  (BIT)  characteristics  and  performance  (npn  &  pnp 
devices;  gated  and  ungated;  all  possible  base  dopings) 

■  Lateral  BJT  matching  (light  base  dopings,  regarded  as  the  most  potentially  useful) 

■  Log-domain  lowpass  and  highpass  filters  (for  very  long  time  constants) 

■  A  Gm-C  lowpass  filter 

■  Miscellaneous  small  building-block  circuits  (e.g.,  differential  pairs,  current  mirrors) 
The  physical  layout  of  this  chip  is  depicted  below  in  Figure  1 : 


Figure  1:  Physical  layout  of  chip_l_sos. 


In  Figure  1,  the  hexagonal  array  at  top  is  the  ESTMD  array,  while  the  lower  central 
portion  contains  individual  test  devices.  Multi-transistor  test  circuits  were  arrayed  around 
the  perimeter,  connecting  to  the  outer  ring  of  bonding  pads.  Because  the  number  of 
bonding  pads  is  limited,  we  addressed  the  I/O  issue  using  a  probe  pad  structure,  in 
conjunction  with  a  probe  card  allowing  simultaneous  contact  to  twenty  probe  pads  for  dc 
testing  purposes.  There  are  six  such  20-pad  structures  for  probing  of  individual  test 
devices  in  the  lower  center  of  the  chip.  With  these  structures,  unpackaged  die  could  be 
placed  on  a  probe  station  for  testing. 
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The  first  bulk  silicon  test  chip,  chip_l_bulk,  was  submitted  to  the  MOSIS  /  Agilent 
fabrication  run  closing  1  July  2002.  This  chip  contained  an  ESTMD  array  similar  to  that 
fabricated  on  the  SOS  chip,  but  modified  for  implementation  in  bulk  silicon.  It  also 
included  test  structures  to  evaluate: 

•  MOS  transistor  matching  (both  standard  pairs  and  common-centroid  quads) 

■  Devices  to  investigate  linear  dimension  effects  on  MOS  transistor  matching  (i.e., 
length  and  width  effects  rather  than  area) 

■  Log-domain  lowpass  and  highpass  filters 

■  A  long-time-constant  lowpass  filter  based  on  linear  current  division  technique  with 
MOS  transistors 

■  A  Gm-C  lowpass  filter 

■  Miscellaneous  small  building-block  circuits  (e.g.,  differential  pairs,  current  mirrors) 
The  physical  layout  of  the  chip  is  depicted  below  in  Figure  2: 


Figure  2:  Physical  layout  of  chip_l_bulk. 


In  Figure  2,  the  hexagonal  array  at  top  is  the  ESTMD  array,  while  the  lower  central 
portion  contains  individual  test  devices.  As  on  chip_l_sos,  multi-transistor  test  circuits 
were  arrayed  around  the  perimeter,  and  probe  pad  structures  were  used  for  individual 
devices.  There  are  three  20-pad  structures  on  chip_l_bulk. 
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2.3.3  STMDChip 

Our  research  plan  called  for  fabrication  of  a  second,  more  advanced  test  chip 
following  the  initial  pair.  Based  on  test  results,  we  decided  that  this  chip  would  be 
fabricated  in  bulk  CMOS  only.  Unfortunately,  access  to  the  Agilent  process  used  for  the 
first  chip  was  discontinued  by  MOSIS  in  the  interim,  and  we  were  forced  to  revert  to  the 
original  TSMC  0.35|Lim  bulk  CMOS  process.  Because  we  had  used  MOSIS  scalable 
design  rules,  translation  of  existing  designs  to  the  new  process  was  not  prohibitively 
difficult.  The  final  chip,  stmd_chip,  was  completed  and  submitted  on  23  June  2003.  It 
contained  two  versions  of  ESTMD  arrays  with  design  revisions  relative  to  the  first 
submssion,  and  an  analog  of  a  wide-field  STMD  cell  that  collates  inputs  from  an  array  of 
ESTMDs.  It  also  included  a  set  of  transistor  matching  test  structures  and  a  revision  of  the 
log-domain  lowpass  filter  appearing  on  the  prior  submission. 

The  physical  layout  of  the  chip  is  depicted  below  in  Figure  3: 


Figure  3:  Physical  layout  of  stmdjchip. 

In  Figure  3,  the  hexagonal  arrays  at  top  left  and  bottom  right  are  the  ESTMD  arrays, 
while  the  central  portion  contains  the  integrated  ESTMD  arrays  and  wide-field  neuron 
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analog.  Further  test  structures  are  placed  around  the  margins  of  the  chip  area  that  lies 
inside  the  bonding  pad  ring. 

2.3.4  Testing 

Following  receipt  of  each  chip  from  the  foundry,  test  and  evaluation  was  performed. 
A  test  board  was  assembled  for  each  chip,  including  a  socket  for  the  device  under  test,  as 
well  as  biasing  and  other  auxiliary  circuitry  needed  to  operate  it.  In  addition,  a 
commercial  field-programmable-gate-array  (FPGA)  integrated  circuit  and  board  were 
procured,  and  the  FPGA  was  programmed  to  generate  control  signals  necessary  to 
dynamically  address  elements  in  the  test  arrays,  in  order  to  supply  inputs  that  simulated 
‘moving  objects’,  and  also  to  generate  control  signals  for  purposes  of  data  acquisition. 

Finally,  a  20-pin  probe  card  and  mounting  hardware  were  procured,  to  allow  contact 
to  and  testing  of  devices  arrayed  in  probe  structures  on  the  first  pair  of  chips.  The  probe 
card  was  mounted  on  the  platen  of  a  probe  station  resident  in  the  Tarmer  Research 
hardware  laboratory.  Testing  was  carried  out  with  standard  laboratory  bench  instruments 
and  computer-controlled  data  acquisition  hardware  at  Tanner  Research. 

3  ACCOMPLISHMENTS  AND  NEW  FINDINGS 

3.1  Experimental  Neurobiology 

We  consider  our  most  significant  accomplishments  in  neurobiology  under  support  of 
this  project  to  be:  characterization  of  the  responses  of  a  range  of  neurons  that  can  be 
considered  STMDs;  development  of  preliminary  classifications  for  some  of  these  cells, 
including  a  small-field  STMD  that  we  have  labeled  an  ‘elementary  small  target 
movement  detector’,  or  ESTMD,  and  which  may  be  an  early  or  intermediate  stage  in  a 
hierarchy  of  STMD  processing;  the  discovery  of  STMD  neurons  capable  of  responding 
selectively  to  small  moving  targets  against  moving  cluttered  backgrounds;  and  the 
finding  of  a  neuron  that  we  believe  will  prove  identifiable  from  individual  to  individual 
of  a  particular  species  (E.  tenax),  according  to  both  physiological  and  anatomical  criteria. 

3.1.1  General  Properties  of  STMD  Neurons 

In  our  initial  experiments,  the  visual  stimuli  applied  to  evaluate  putative  ESTMD 
neurons  consisted  of  small  targets  moving  against  homogeneous  static  backgrounds,  and 
additionally  in  control  experiments,  larger  moving  objects  and  moving  gratings.  In  these 
experiments,  we  found  that  STMD  neurons  tend  to  be  ‘spiking’  or  action-potential- 
generating  cells,  and  that  a  variety  of  neurons,  with  a  range  of  receptive  field  properties, 
could  be  found  in  all  of  the  subject  species.  (In  addition  to  the  species  studied 
experimentally,  O’Carroll  and  Henry  also  reviewed  prior  data  obtained  from  putative 
STMD  cells  in  a  mantid,  a  predatory  species  from  a  third  insect  order,  and  foimd  similar 
characteristics  in  those  cells  as  well.)  Some  STMD  neurons  have  small  excitatory 
receptive  fields  (subtending  a  few  degrees  to  perhaps  ten  degrees  of  visual  angle),  and 
some  much  larger  receptive  fields,  from  a  few  tens  of  degrees  to  the  entire  hemi-field 
viewed  by  an  eye.  The  cells  with  largest  receptive  fields  tend  to  have  the  most  'highly 
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tuned'  response  properties,  characterized  by  no  spontaneous  firing  activity,  and  no 
response  to  'non-preferred'  targets  such  as  bars,  edges  or  grating  patterns.  These  cells  do, 
however,  respond  very  strongly  to  small  targets  (down  to  0.5  x  0.5  degrees)  that  move  at 
any  location  within  the  receptive  field.  The  cells  with  smaller  receptive  fields  typically 
display  distinct  response  maxima  at  the  center  of  those  fields.  These  neurons  tend  to  be 
slightly  less  selective  for  small  target  motion,  giving  weak  (transient)  responses  to  a 
variety  of  other  stimuli.  Interestingly,  when  the  graded  memebrane  potentials  of  these 
neurons  are  observed,  they  show  evidence  of  a  complex  receptive  field  structure,  with  a 
much  larger,  asymmetric  inhibitory  'surround'  region  in  addition  to  the  excitatory 
‘center’.  We  also  found  some  evidence,  in  the  form  of  recordings  from  cells  with  similar 
response  properties  but  spatially  offset  receptive  fields,  that  individual  neurons 
(particularly  the  smaller-field  cells)  may  be  retinotopically  organized. 

The  general  properties  of  STMD  neurons  deduced  from  these  experiments  include: 

1.  Long  response  latency  (typically  ca.  40-50  ms). 

2.  Strong  selectivity  for  small  targets. 

3.  Both  direction  selective  and  non-directional  responses  to  target  motion. 

4.  Variety  of  receptive  field  sizes. 

5.  Differences  in  velocity  tuning:  some  sharply  tuned  for  maximal  response  to  a 
particular  velocity,  some  monotonic. 

6.  Tendency  for  target  direction  selectivity  in  velocity  tuned  cells. 

7.  Direct  relation  between  velocity  optimum  and  target  extent  in  the  direction  of 
motion,  in  velocity  tuned  cells. 

8.  Cells  with  largest  receptive  fields  tend  to  have  strongest  selectivity  for  size, 
direction,  and  tend  to  lack  any  spontaneous  activity. 

9.  Post-synaptic  noise  evident  in  some  cells:  IPSPs  occur  as  lateral  target  extent  is 
increased,  with  noisy  ‘breakthrough’  EPSPs,  suggesting  ‘competitive’ 
recruitment  of  both  excitatory  and  inhibitory  pathways. 

10.  Post-synaptic  noise  is  suppressed  by  wide-field  motion  (possibly  pre-synaptic 
gating),  suggesting  more  than  one  source  of  inhibitory  feedback. 

3.1.2  Wide-Field  STMDS 

STMD  neurons  were  found  to  fall  into  two  clear  groups  in  terms  of  their  tuning  to 
target  velocity:  those  with  a  distinct  optimum  speed  (‘velocity  tuned’)  and  those  which 
give  monotonically  increasing  responses  up  to  the  highest  speeds  tested  (ca.  1000  degrees 
per  second).  Neither  class  of  velocity  tuning  is  an  obvious  emergent  property  from  the 
array  of  elementary  motion  detectors  that  are  assumed  to  form  inputs  to  the  STMDs,  and 
they  likely  result  from  a  combination  of  the  inherent  velocity  tuning  of  EMDs  and  the 
d5mamic  properties  associated  with  the  receptive  fields  of  STMDs  themselves.  This  is 
further  suggested  by  results  of  a  similar  experiment  where  we  examined  velocity  tuning 
of  an  HS  neuron  (a  wide-field  motion  detector)  in  response  to  small  moving  targets. 
Although  they  respond  more  weakly  to  this  stimulus  than  to  wide-field  motion,  non¬ 
linear  spatial  summation  within  their  receptive  fields  (sometimes  called  'gain  control') 
means  that  they  produce  measurable  responses  to  moderate  sized  targets  (3  by  3  degrees). 
Surprisingly,  the  HS  neuron  shows  no  distinct  velocity  optimum,  instead  responding 
monotonically  to  increasing  target  speed  to  the  highest  velocities  tested  (ca.  300  degrees 
per  second).  By  comparison,  direction  selective  STMDs  often  (if  not  always)  show  a  very 
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sharp  peak  in  velocity  tuning  at  much  lower  speeds.  Furthermore,  while  STMD  neurons 
show  tuning  that  shifts  to  higher  speeds  as  the  longitudinal  dimension  of  the  target 
increases,  the  HS  neuron  does  not. 

The  velocity-tuned  wide-field  STMD  neurons  can  be  further  classified  on  the  basis  of 
their  direction  selectivity  and  velocity  tuning  into  several  sub-classes: 

(1)  Strongly  direction-selective  neurons  with  a  low  optimum  target  speed  (ca.  30 
degrees  per  second).  [This  class  has  been  clearly  identified  in  Hemicordulia,  and  we  have 
preliminary  evidence  for  these  neurons  in  Aeshnd]. 

(2)  Strongly  direction-selective  neurons  with  a  high  optimum  target  speed  (ca.  100 
degrees  per  second  or  more).  [This  class  has  been  clearly  identified  in  Aeshna,  and  we 
have  preliminary  evidence  for  these  neurons  in  Hemicordulia]. 

(3)  Weakly  direction-selective  (or  non-selective)  neurons  with  a  high  optimum  target 
speed  (ca.  100  degrees  per  second  or  more).  These  cells  tend  to  be  slightly  less  selective 
for  small  target  size  than  strongly  direction-selective  neurons.  [This  class  has  been 
clearly  identified  in  both  species]. 

We  found  no  evidence  for  non-directional  neurons  with  low  velocity  optima. 

Experimental  evidence  with  respect  to  these  characteristics  is  not  as  complete  for 
Eristalis  as  for  the  dragonfly  species,  but  results  so  far  are  consistent  with  the  same 
conclusions. 

Some  examples  of  response  characteristics  from  a  wide-field,  velocity-tuned  STMD 
are  depicted  below  in  Figure  4.  These  results  depict,  on  the  left,  a  direct  relationship 
between  longitudinal  target  size  (i.e.,  extent  in  the  direction  of  motion)  and  the  optimum 
response  velocity,  and  on  the  right,  the  selectivity  of  the  cell  for  smallness  of  the  target  in 
the  lateral  direction  (i.e.,  extent  perpendicular  to  the  direction  of  motion). 
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Speed  (7s) 

Figure  4:  Velocity  tuning  at  different  stimulus  sizes  and  target  speeds.  At  left, 
response  of  an  STMD  in  the  dragonfly  Hemicordulia  tau  to  a  small  moving  target  as 
a  function  of  target  velocity,  with  the  extent  of  the  target  in  the  direction  of  motion 
(the  ‘horizontal’  direction)  as  a  parameter.  As  the  target  becomes  longer,  the 
velocity  tuning  curves  shift  to  higher  optima  At  right,  similar  velocity  tuning  curves 
for  an  STMD  in  tau ,  but  with  lateral  size  of  the  target  (i.e.,  extent  perpendicular 
to  the  direction  of  motion)  as  a  parameter.  Lateral  extent  has  no  effect  on  tuning 
optima  of  the  cell,  only  its  general  response  level. 

The  clear  evidence  for  at  least  two  velocity  optima  among  the  direction-selective 
STMDs  is  an  exciting  finding.  We  propose  that  such  a  finding  would  be  consistent  with  a 
possible  role  for  the  neurons  in  'velocity  parallax'  -  encoding  target  distance  on  the  basis 
of  apparent  speed  of  targets  during  translatory  motion  by  the  dragonfly.  Both  Aeshna  and 
Hemicordulia  are  so-called  'hawking'  dragonflies,  which  initially  detect  target  motion 
during  hovering  flight  and  then  pursue  prey  aerially.  Following  detection  of  prey  during 
hovering,  these  insects  make  a  characteristic  translation  at  moderate  speed,  before 
attempting  target  interception.  Diuing  such  translations,  the  two  direction-selective 
STMD  classes  described  above  would  respond  differentially  to  the  same  target,  even  if 
that  target  were  stationary,  in  a  manner  that  depends  only  on  target  distance.  A  'higher 
order'  neuron  could  potentially  integrate  information  from  two  or  more  of  such  neurons  to 
produce  a  response  that  is  velocity  opponent  (and  thus  distance  opponent  during  self- 
motion). 

3. 1.3  The  Putative  Elementary  Small  Target  Movement  Detector 

Further  work  and  characterization  of  small-field  STMDs  led  us  to  the  conclusion  that 
they  might  constitute  a  basic  unit  for  small-target  motion  processing,  an  'elementary 
small  target  motion  detector'  or  ESTMD,  perhaps  serially  hierarchical  (i.e.,  pre-synaptic) 
to  wide-field  'collator'  cells  corresponding  to  the  wide-field  STMD  neurons.  The 
receptive  fields  of  putative  ESTMD  cells  display  very  small  excitatory  centers,  with 
larger  and  more  diffuse  inhibitory  surrounds.  The  excitatory  centers  suggest  that  these 
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cells  may  receive  excitatory  inputs  from  only  a  few  adjacent  ommatidia  (presumably  via 
EMDs  receiving  input  from  those  ommatidia).  They  typically  have  a  much  larger, 
asymmetric  (offset  or  notched)  inhibitory  surround  region,  visible  only  in  the  graded 
potential  of  the  cell  as  it  is  subject  to  small  moving  target  stimuli.  These  cells  tend  to  be 
direction  selective,  with  a  preferred  direction  always  corresponding  to  the  orientation  of 
the  offset  surround:  only  a  target  that  enters  the  receptive  field  through  the  ‘notch’  or 
weakest  part  of  the  inhibitory  region  is  capable  of  eliciting  sufficient  excitation  to  cause 
the  firing  of  a  burst  of  action  potentials. 

An  example  of  the  receptive  field  of  a  putative  ESTMD  is  shown  below  in  Figure  5, 
recorded  intracellularly  from  the  lobula  of  a  hoverfly  Eristalis.  In  this  plot,  each 
rectangular  sub-field  represents  the  boundaries  of  the  screen  on  which  stimuli  (small 
drifting  targets)  were  displayed.  The  response  of  the  neuron  to  targets  moving  from  left  to 
right  (left  panels)  and  then  right  to  left  (right  panels)  at  a  series  of  vertical  positions  on 
the  screen  (presented  in  random  order)  is  plotted,  with  the  vertical  axis  representing 
space  and  the  horizontal  axis  time.  The  time  axis  is  scaled  equivalently  to  the  spatial 
dimension,  so  that  flipping  the  right  panels  about  their  vertical  axes  transforms  the  data 
back  into  space. 
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Figure  5:  Receptive  field  plots  for  an  STMD  in  the  hoverfly  Eristalis  tenax. 
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In  each  of  the  plots  in  Figure  5,  the  upper  panel  is  the  graded  (sub-threshold) 
response  of  the  neuron,  obtained  by  digitally  filtering  out  action  potentials  from  each  raw 
response  recorded,  while  the  lower  plots  are  averaged  firing  rates  obtained  by  counting 
numbers  of  action  potentials  generated  during  stimulus  presentation.  Because  these 
neurons  t5^ically  lack  spontaneous  firing  activity,  it  is  only  the  graded  plots  that  reveal 
the  inhibitory  sub-regions  of  the  receptive  field,  exemplifying  the  primary  advantage  of 
the  (technically  demanding)  single  cell  intracellular  recording  technique  used.  Figure  6 
below  illustrates  similar  data,  in  this  case  for  single  passes  of  a  target  through  the 
receptive  field  of  a  cell. 


Ta  rget  motion  across  excitatory  field  Target  motion  across  inhibitory  irei  d 


Figure  6:  ESTMD  responses  to  moving  stimuli  aligned  with  preferred  direction,  in 
both  directions  of  motions  and  different  locations  within  the  receptive  field. 

In  the  plot  at  left,  the  target  is  drifted  left  to  right  (black  trace)  and  then  right  to  left  (blue) 
at  the  receptive  field  center  of  a  neuron  with  the  ‘notch’  on  the  right  hand  side.  The 
pictogram  in  the  top  right  of  each  figure  gives  a  legend  and  approximate  representation  of 
the  receptive  field  organization,  with  red  outlining  an  excitatory  region,  and  blue,  an 
inhibitory  region.  Analyzing  the  receptive  fields  of  a  number  of  putative  ESTMD 
neurons,  we  found  that  the  notch  in  the  inhibitory  subfield  always  corresponds  to  the 
preferred  direction  of  the  ESTMD  neuron. 

One  notable  feature  seen  in  plots  of  the  graded  membrane  potential  is  that  the 
inhibitory  gain  to  small  target  motion  is  comparable  to  the  gain  of  the  much  smaller 
excitatory  region,  producing  in  some  cases  up  to  8  or  even  lOmV  hyperpolarization. 
Furthermore,  in  the  neurons  illustrated,  the  central  excitatory  region,  although  it  shows 
some  apparent  response  for  targets  moving  in  either  direction,  also  shows  evidence  for  at 
least  weak  direction  selectivity  (consistent  with  the  hypothesis  that  it  receives  input  from 
EMDs  of  the  Reichardt  type),  while  the  inhibitory  surround  does  not.  The  very  high  gain 
in  both  regions,  combined  with  these  other  properties  suggests  a  possible  basis  for 
selectivity  for  small  targets  in  the  lateral  dimension,  since  a  wide  variety  of  other  stimuli 
would  inevitably  stimulate  both  excitatory  and  inhibitory  receptive  field  sub-regions  at 
once,  producing  weak  or  absent  spiking  responses  in  the  output. 

3.1.4  The  ‘Moving  Target /Moving  Background’ Property 

•  With  the  increasing  sophistication  of  stimulus  routines  that  were  developed  for  the 
Vision  Egg,  we  were  able  in  the  final  year  of  the  project  to  present  stimuli  that  contained 
small  moving  targets  against  moving  background  patterns.  Experiments  performed  with 
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such  stimuli  led  to  a  significant  new  finding;  the  capability  of  some  STMD  neurons  in 
both  dragonflies  and  hoverflies  to  discriminate  such  small  moving  targets  against  moving 
backgrounds,  even  when  the  target  is  otherwise  indistinguishable  from  local  features  in 
that  baekground.  For  brevity,  we  refer  to  this  capability  as  the  ‘moving  target  /  moving 
background’  property. 

The  best-documented  case  of  this  property  has  been  in  a  cell  in  Eristalis  that  appears 
to  be  a  stereotypical  neuron  that  is  present  from  individual  to  individual.  This  cell  is 
selective  for  direction  of  target  travel,  and  it  appears  to  be  capable  of  detecting  small 
object  motion  for  any  combination  of  relative  velocity  and  contrast  that  permits  target 
discrimination  by  the  human  eye  and  brain.  We  were  unable  find  a  background  pattern 
that  completely  abolished  its  response  to  the  moving  target,  providing  the  target 
continued  to  move  within  the  receptive  field  and  in  the  preferred  direction.  It  completely 
rejects  backgroimd  motion  alone. 

The  following  figure  illustrates  a  typical  response.  In  this  case,  the  response  was 
recorded  to  a  control  stimulus  (blank  screen  of  mean  luminance)  for  1  second  before  and 
after  a  2  second  period  during  which  both  figure  and  background  moved  across  the 
receptive  field. 


target  trajectory  as  function  of  time 


pre-stimulus  period  peri-stimulus  period  post-stimulus  period 

(blank  screen)  (moving  target  and  background)  (blank  screen) 

Figure  7:  Response  by  an  STMD  neuron  in  Eristalis  to  a  small  moving  target 
(circular  motion)  in  the  presence  of  moving  background  clutter. 


The  background  consisted  of  random  texture  elements  (textels),  each  subtending 
approximately  one  degree  in  visual  space,  and  distorted  onto  a  cylindrical  projection  of 
space  around  the  insect.  A  target  moved  against  this  background  in  a  circular  motion, 
thus  coming  into  and  out  of  phase  with  direction  of  the  background  motion,  although 
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there  remained  a  speed  differential  at  all  times  in  this  experiment.  In  the  figure  shown, 
the  target  moved  through  just  less  than  4  complete  rotations  during  the  peri-stimulus 
period.  The  neuron  is  entirely  ‘silent’  whenever  the  screen  is  blank,  or  background 
motion  is  presented  alone  (not  shown).  The  response  pattern  includes  powerful  bursts  of 
action  potentials  (spikes)  whenever  the  target  moves  in  a  specific  direction  across  the 
receptive  field,  although  the  response  is  weakly  inhibited  by  motion  in  the  opposite 
direction. 

This  direction  selective  response,  previously  observed  in  many  STMD  recordings 
against  blank  backgrounds,  is  evident  if  the  data  from  a  similar  recording  are  re-plotted 
on  polar  coordinates.  In  Figure  8  below,  the  location  of  individual  spikes  recorded  during 
the  peri-stimulus  period  is  shown  by  a  colored  circle,  with  a  different  color  for  each  of 
the  4  cycles  through  which  the  target  moves.  In  each  case,  spikes  were  only  recorded 
when  the  target  moves  downward.  We  have  recorded  other  cells  in  which  the  response 
may  be  to  other  preferred  directions. 
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Figure  8:  Individual  spikes  plotted  in  polar  coordinates  highlighting  direction-selective 
response.  Spikes  were  only  observed  when  the  target  was  moving  downward. 

We  find  that  the  observed  response  is  extremely  robust  against  the  direction  or 
structure  of  the  background  optic  flow  pattern.  Only  when  the  target  size  is  similar  to  the 
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structure  of  the  background  features,  of  similar  contrast  and  moving  at  the  same  speed,  in 
the  same  direction  does  the  response  become  completely  abolished. 

Figure  9  below  illustrates  the  stimulus  and  response  used  in  Figure  7  above,  along 
with  the  response  to  a  second  stimulus  with  a  higher  background  contrast.  In  each  case, 
the  pictogram  at  left  shows  a  screenshot  from  the  actual  stimulus  sequence.  In  the  high 
contrast  stimulus,  the  target  is  smaller  and  blends  almost  perfectly  into  the  background  of 
this  still  frame,  while  the  target  in  the  upper  plot  is  more  clearly  visible  (upper  right  of 
the  image).  During  motion,  the  small  target  in  the  high  contrast  background  is  visible  to 
human  observers,  but  is  much  more  difficult  to  make  out  than  the  larger  target  in  the  low- 
contrast  background.  Similarly,  the  response  from  the  neuron  is  weaker  in  case  of  the 
small  target,  although  it  still  shows  clear  direction  selectivity.  These  results  suggest  that 
the  individual  neurons  studied  display  selectivity  for  target  motion  in  clutter  similar  to 
that  found  in  the  human  visual  system:  As  a  general  rule,  our  observation  is  that  the 
limit?  of  both  systems  are  similar:  if  a  human  can’t  see  the  target  on  the  screen,  the  insect 
neuron  does  not  fire  either. 


large'  target  (approx.  3  degrees),  low  contrast  background 


pre-stimulus  period  peri-stimulus  period  post-stimulus  period 

(blank  screen)  (moving  target  and  background)  (blank  screen) 

Figure  9:  Stimulus  and  response  for  a  large  target  and  low  contrast  background 
(top)  and  a  small  target  and  high  contrast  background  (bottom). 

Because  it  is  so  difficult  to  visualize  these  stimuli  in  still  frame  representations,  we 
have  constructed  movie  files  depicting  them.  One  such  file  is  included  herewith  on  the 
CD  used  for  submission  of  this  report. 
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Because  we  have  on  multiple  occasions  penetrated  cells  in  different  specimens  of 
Eristalis  that  have  response  and  receptive  field  characteristics  (including  the  moving 
target  /  moving  background  property)  essentially  the  same  as  that  reported  above,  and 
these  cells  have  all  been  located  in  the  same  area  of  the  lobula,  we  believe  that  we  have 
recorded  from  a  stereotjtpical  and  identifiable  neuron.  In  the  final  quarter  of  this  contract, 
Irene  Moyer,  a  summer  student  working  in  our  laboratory,  has  completed  a  4  week 
project  to  develop  these  dye  filling  techniques.  We  have  procured  3  dyes,  Lucifer  Yellow 
CH,  Cascade  Blue  and  Alexa  Fluor  350.  We  have  successfully  filled  several  tangential 
cells  in  Eristalis  with  Lucifer  Yellow,  although  not  the  candidate  STMD  neuron  yet.  We 
are  presently  trialing  a  very  new  dye,  Alexa  fluor  350,  which  holds  great  promise  for  this 
work,  since  the  recording  properties  of  electrodes  filled  with  this  dye  are  only  slightly 
worse  than  electrodes  filled  with  simple  electrolyte  and  much  better  than  with  Lucifer 
Yellow  CH  (which  has  been  our  traditional  dye  of  choice).  Alexa  also  has  much  higher 
fluorescence  and  so  promises  to  be  better  suited  to  shorter  recordings  from  smaller  cells 
than  anything  we  have  used  previously. 

3.2  Modeling  and  Simulation 

In  our  modeling  work,  we  examined  a  range  of  processing  relevant  to  STMD 
responses.  During  the  first  half  of  the  project,  we  focused  on  the  small-field  STMD 
neurons  that  we  putatively  labeled  ESTMDs,  and  examined  in  some  detail  the  responses 
of  elementary  motion  detectors  (EMDs)  to  small  target  motion.  This  led  later  in  the 
project  to  concepts  for  processing  that  could  endow  STMD  neurons  with  selectivity  for 
small  target  size  in  the  direction  of  motion.  Finally,  we  also  examined  possible 
mechanisms  for  collation  of  ESTMD  outputs,  and  revised  our  putative  hierarchy  of 
STMD  processing  in  view  of  the  discovery  of  the  ‘moving  target  /  moving  backgroimd’ 
property. 

3.2.1  ESTMD  Modeling  '■ 

Study  of  the  characteristics  of  putative  ESTMD  neurons  led  us  to  propose  a 
conceptual  model  for  how  they  might  operate.  The  essential  features  of  this  concept  are: 
1)  a  center-surroimd  receptive  field  organization  with  sensitivity  to  moving  objects  in  the 
visual  field,  and,  as  noted  in  Section  3.1.3,  a  small  excitatory  center  and  a  spatially  offset, 
or  ‘notched’,  inhibitory  surround;  2)  nonlinear  dynamic  properties,  leading  to  persistence 
of  inhibition  or  excitation  following  the  passage  of  a  target  across  a  particular  point  in  the 
receptive  field.  Inputs  to  the  ESTMD  were  assumed  to  be  derived  from  elementary 
motion  detectors. 

The  particular  receptive  field  characteristics  (much  as  in  complex  cells  in  vertebrate 
visual  cortex)  are  assumed  to  be  formed  by  competitive  integration  of  excitatory  and 
inhibitory  inputs,  with  a  small  (ca.  5-10  degree)  region  in  which  excitation  dominates  the 
response,  and  a  much  larger  region  dominated  by  inhibition.  However,  unlike  the 
complex  cell,  in  which  the  two  regions  are  coaxial,  the  inhibitory  region  in  the  ESTMD  is 
spatially  offset  relative  to  the  excitatory  region.  The  superposition  of  inhibition  and 
excitation  produces  the  characteristic  appearance  of  a  ‘horseshoe-shaped’  inhibitory 
surroimd.  We  found  that  phenomenologically,  such  receptive  fields  could  be  modeled 
well  by  a  difference-of-Gaussians  function,  in  which  the  negative  or  inhibitory  Gaussian 
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center  is  offset  in  visual  space  with  respect  to  the  positive  or  excitatory  component  (see 
Figure  10  below). 


Figure  10;  A  fit  (middle  plot)  using  a  differenc^of-Gaussians  functon,  to  a  typical 
data  set  representing  an  ESTMD  receptive  field  (lower  plot).  The  upper  plot  is  the 
residual  error. 

This  model  is  particularly  interesting  among  alternatives  because  the  complex  cell 
model  has  previously  been  shown  to  have  widespread  applicability  to  the  earliest  stages 
of  visual  processing  (‘spatial  antagonism’  or  high  pass  filtering)  in  systems  as  diverse  as 
inseet  vision  and  mammalian  retinal  processing.  It  is  thus  plausible  to  suggest  that  the 
neural  processing  of  ESTMDs  might  have  evolved  initially  from  complex  cell  analogues 
that  operated  on  EMD  outputs  to  ‘sharpen’  spatial  location  of  moving  features,  and  which 
produced,  as  a  consequence,  basie  selectivity  for  small  targets.  Subsequent  selection  for 
neural  pathways  eapable  of  tracking  moving  targets  may  have  led  to  the  fine-tuning  of 
these  receptive  fields  into  the  spatially  offset  fields  that  we  see  in  our  present  data. 

In  this  model,  the  open-ended  ‘horseshoe’  shape  of  the  inhibitory  subfield,  in 
combination  with  the  tonic  or  persistent  character  of  the  neural  responses,  serves  several 
functions.  The  ‘notch’  provides  a  means  by  which  targets  moving  in  the  preferred 
direction  can  enter  the  receptive  field  of  the  ESTMD  without  recruiting  significant 
inhibition,  whereas  motion  toward  the  center  from  other  directions  recruits  tonic 
inhibition  which  depresses  excitation  as  the  target  crosses  the  center.  Furthermore, 
motion  in  the  optimal  direction  recruits  tonic  excitation  that  outlasts  the  passage  aeross 
the  excitatory  center.  Finally,  a  visual  object  whose  lateral  width  is  greater  than  the  width 
of  the  ‘notch’  in  the  inhibitory  field  would  recruit  inhibition  even  if  it  traveled  in  the 
preferred  direction.  The  ESTMD  as  modeled  thus  contributes  both  to  the  properties  of 
directional  tuning  and  selectivity  for  laterally  small  targets. 

We  suggest  that  such  ESTMDs  may  be  organized  retinotopically,  with  overlapping 
receptive  fields  (as  in  the  following  diagram,  where  excitation  is  shown  red,  and 
inhibition  in  blue,  and  only  a  single  one-dimensional  row  is  depicted  for  clarity): 
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Excitation  outlasting  the  passage  of  the  target  across  each  center  of  such  an  array 
would  allow  a  cumulative  build-up  of  powerful  excitation  (positive  reinforcement)  in  a 
post-synaptic  collator  neuron. 

This  model  makes  several  clear  predictions: 

1.  If  we  record  responses  to  targets  drifting  across  the  excitatory  center,  in  both  the 
preferred  and  anti-preferred  directions,  delayed  excitation  should  lead  to  asymmetry 
in  the  strength  of  the  response  to  target  motion  as  it  crosses  the  inhibitory  ‘flank’  of 
the  ESTMD,  even  though  the  flank  itself  is  not  inherently  direction  selective.  This 
hypothesis  has  been  tested  and  verified  experimentally  by  results  like  that  shown  in 
Figure  6  in  Section  3.1.3.  In  that  figure,  excitation  by  preferred  direction  target 
motion  outlasts  the  relatively  brief  crossing  of  the  excitatory  center  and  limits 
inhibition  by  the  left-most  inhibitory  part  of  the  receptive  field  to  negative  potentials 
considerably  smaller  in  magnitue  than  those  elicited  by  a  crossing  of  the  same 
inhibitory  region  in  the  opppsite  direction. 

2.  The  asymmetry  in  inhibitory  gain  sho^d  disappear  if  we  were  to  repeat  the 
experiment  with  the  target  appearing  within  a  limited  portion  of  the  receptive  field, 
e.g.,  if  it  moves  only  within  the  inhibitory  subfield  (thus  altering  the  recent  stimulus 
history). 

3.  This  model  predicts  a  long  response  latency  in  the  ESTMD  and  in  downstream 
STMDs,  due  to  the  tonic  nature  of  the  response  (a  form  of  non-linear  low-pass 
filtering)  and  the  positive  reinforcement  produced  by  targets  crossing  sequential 
receptive  field  of  ESTMDs.  As  noted  earlier,  the  typical  latency  in  response  of 
STMD  neurons  is  much  longer  than  that  of  other  insect  tangential  neurons  -  of  the 
order  of  40-50  ms,  compared  with  12  ms  in  an  HS  neuron  (wide-field  movement 
detector). 
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4.  A  final  prediction  is  that  delayed  inhibition  and  excitation  within  the  ESTMD  would 
alter  the  velocity  tuning  of  a  collator  (STMD)  cell  radically  compared  with  that  of  a 
simple  EMD  collator  like  an  HS  neuron.  Although  it  is  premature  to  attribute  this 
difference  to  the  kinetics  of  ESTMD  receptive  field  interactions,  this  seems  a  likely 
source. 

In  considering  how  such  an  ESTMD  might  be  implemented  physically,  we  sought  a 
model  that  is:  both  neurobologically  plausible,  and  a  reasonable  a  basis  for  an  artificial 
circuit  analog;  consistent  with  a  retinotopic  distribution;  and  in  some  sense  based  on  a 
minimally  complex  architecture.  Such  a  physical  model  is  illustrated  below  in  Figure  12, 
in  a  one-dimensional  example  for  sake  of  clarity: 


EXCITATORY 

INTERNEURON 


INPUT 
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DIRECTIONAL 
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LATERAL  INHIBITORY 
NETWORK  (DIFFUSIVE) 


Figure  12:  ESTMD  architecture  (one-dimensionai  array  for  illustration). 

Two  major  simplifying  assumptions  are  made  in  this  model:  one,  the  temporal 
characteristics  are  attributed  to  a  single  ‘excitatory  intemeuron’  per  ESTMD  unit  cell, 
and  two,  only  local  interconnections  are  made  between  ESTMD  units  (although  signals 
can  propagate  over  larger  distances  by  diffusion).  The  dynamic  characteristics  of  the 
‘excitatory  intemeuron’  comprise  a  fast  onset  and  slow  decay  of  excitation  due  to  input 
from  peripheral  stages  in  the  visual  motion  pathway.  This  ‘intemeuron’  directly  excites 
an  ESTMD  output  unit,  and  also  provides  input  to  a  diffusive  network  that  is  used  to 
propagate  inhibition.  (This  arrangement  eliminates  the  need  for  two  time-dependent 
units,  one  for  excitation  and  one  for  inhibition.)  Although  signal  propagation  between 
inhibitory  units  is  diffusive,  these  units  actively  inhibit  neighboring  output  units  in  a 
directional  fashion:  in  the  one-dimensional  example  of  Figure  12,  inhibition  is 
propagated  from  an  inhibitory  neuron  onto  the  output  unit  to  its  right,  but  not  the  one  to 
its  left.  This  particular  architecture  is  therefore  tuned  to  leftward  motion.  The  same 
diffusive  network  could  be  used  to  form  ESTMDs  with  other  directional  tunings,  by 
virtue  of  different  patterns  of  active  inhibitory  connections  onto  different  output  rmits. 

Although  the  example  above  is  one-dimensional,  we  simulated  ESTMDs  arranged  in 
two-dimensional  hexagonal  retinotopic  arrays,  in  an  architecture  in  which  active 
inhibition  impinges  on  an  ESTMD  output  unit  from  five  out  of  six  nearest  neighbors.  The 


Output  (arbitrary  units) 
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preferred  direction  for  excitation  is  thus  the  direction  from  which  active  inhibition  does 
not  impinge  on  the  ESTMD  unit.  (A  stimulus  moving  in  that  direction  does  cause  some 
initial  inhibition  due  to  diffusive  propagation  to  neighboring  cells  that  do  have  inhibitory 
coimections  to  the  target  cell).  Although  excitatory  receptive  fields  of  ESTMDs  are 
typically  several  ommatidia  wide  in  visual  space,  we  modeled  each  ESTMDs  with  a 
single  excitatory  input.  Thus  our  models  could  be  thought  of  as  either  a  ‘spatially 
compressed’  ESTMD,  or  as  operating  on  summed  input  signals  from  a  number  of 
adjacent  EMDs. 

A  61 -unit  hexagonal  array  of  ESTMD  cells  was  selected  for  simulation,  and 
ultimately  for  implementation  in  analog  silicon.  Simulations  were  performed  using  the 
tool  SPICE.  The  temporal  properties  were  based  on  the  response  of  a  nonlinear  circuit 
(described  in  detail  in  Section  3.3.2  below)  with  rapid  charge  and  slow  discharge 
characteristics.  The  diffusive  network  was  modeled  with  a  resistive  grid,  in  which  the 
spreading  or  space  constant  is  determined  by  the  ratio  of  shunt  to  lateral  resistances 
(actually  implemented  with  active  devices).  Inputs  were  assumed  to  constitute  moving 
pulses  corresponding  to  the  retinotopic  output  of  earlier  stages  of  motion  processing,  as 
induced  by  passage  of  a  small  moving  target.  With  appropriately  chosen  scaling  of 
inhibition  and  excitation,  responses  such  as  that  depicted  below  in  Figure  13  were 
obtained  from  these  simulations. 


Abstract  ESTM  D  Arr  ) 


Figure  13:  Abstract  ESTMD  array  response  to  a  ‘moving  target’  crossing  the 
excitatory  center,  in  both  the  preferred  (green)  and  opposite  or  antipreferred  (red) 

directions. 


Contract  F49620-01-C-0030  Final  Renort 


26 


These  responses  may  be  compared  to  the  actual  data  taken  from  an  ESTMD  neuron,  in 
the  left-side  panel  of  Figure  6. 

3.2.2  Selectivity  for  Longitudinally  Small  Targets 

During  our  simulation  studies,  one  topic  that  we  investigated  was  the  response  to 
small  moving  targets  of  the  elementary  motion  detector  itself,  the  presumed  front-end  for 
small-target  detection.  These  efforts  ultimately  led  to  a  model  for  selectivity  for 
smallness  of  a  moving  visual  object  in  the  longitudinal  dimension  (direction  of  travel). 

This  model  was  built  upon  observations  of  how  EMD  responses  to  moving  targets 
depend  on  the  characteristics  of  temporal  filtering  that  is  assumed  to  take  place  in  early 
vision.  Basic  correlational  EMDs  were  simulated,  with  inputs  constituting  moving  targets 
with  constant  contrast  with  background,  and  of  various  longitudinal  extents.  We  found 
that  when  the  early  vision  ‘prefiltering’  passes  a  DC  component  (as  with  a  lowpass  filter) 
the  EMD  response  becomes  triphasic  as  the  target  extent  becomes  small.  Prefiltering  that 
passes  no  DC  (such  as  a  bandpass  filter)  does  not  result  in  such  triphasic  EMD  outputs. 
This  is  illustrated  below  in  Figure  14,  which  shows  the  response  of  an  EMD  to  targets  of 
four  different  sizes  moving  across  its  receptive  field,  for  filters  of  each  type: 


Perfect  low-pass  pre-filter  (Log-normal)  Perfect  band-pass  pre-filter  (Difference  of  Log-normals) 


Figure  14:  Response  of  a  correlational  EMD  to  passage  of  a  moving  target  of 
different  longitudinal  extents.  Left  response  is  with  lowpass  temporal  prefiltering, 
and  right,  with  bandpass  temporal  prefiltering.  Abscissa  is  time  in  ms  and  ordinate, 

EMD  output  in  arbitrary  units.  Target  speed  is  40°/s.  The  'perfect  low-pass',  filter 
used  a  log-normal  impulse  response  with  10ms  time  to  peak  and  sigma=0.4.  'Perfect 
band-pass'  used  a  difference  of  this  (gain  unity)  and  a  second  log-normal  with  the 
same  sigma  but  30ms  time  to  peak,  resulting  in  an  impulse  response  with  zero  mean. 
Longitudinal  target  extent  is  a  parameter  in  each  case. 

This  model  used  1  degree  inter-receptor  angle,  1 .4  degree  (half-width)  Gaussian  blur 
(2  dimensions),  100  ms  pre-stimulus  period  (no  target  motion),  dark  target  (luminance  0) 
on  a  bright  background  (1.0),  with  a  first-order  (exponential)  delay  filter  (time 
constant=35  ms).  The  ‘double  log  normal’  impulses  responses  used  for  the  prefilters  have 
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been  previously  shown  to  give  excellent  fits  to  first  order  kernels  measured  in  fly  lamina 
monopolar  cells  (LMCs)  with  white-noise  techniques  ].  The  LMCs  are  likely  presynaptic 
elements  to  the  EMDs,  and  their  responses  reflect  both  their  own  highpass  dynamic 
characteristics  and  the  lognormal  lowpass  characteristic  of  the  photoreceptors  which 
supply  their  inputs. 

LMCs,  however,  are  actually  ‘imperfect’  high  pass  filters,  which  typically  transmit  a 
weak  DC  response,  and  indeed  become  strictly  low-pass  at  lower  light  levels.  To  model 
this  feature,  we  altered  the  weight  of  the  high  pass  component  of  the  double  lognormal, 
which  generates  impulse  responses  with  small  negative  or  positive  means,  and  in  the 
frequency  domain  leads  to  passage  of  small  inverted  or  noninverted  DC  components, 
respectively.  Figure  15  below  shows  model  outputs  for  ‘imbalanced’  bandpass  filters 
based  on  this  imperfect  LMC  highpass  characteristic: 


Band-pass  pre-filter  with  20%  DC  (positive)  Band-pass  pre-filter  with  20%  DC  (negative) 


Figure  15:  Response  of  a  correlational  EMD  to  passage  of  a  moving  target  of 
different  longitudinal  extents.  Left  response  is  with  bandpass  temporal  prefiltering 
that  passes  20%  noninverted  DC,  and  right,  with  bandpass  temporal  prefiltering 
that  passes  20%  inverted  DC.  Other  parameters  as  in  Figure  14. 


In  these  two  'imperfect'  variants,  the  second  log  normal  part  of  the  impulse  function 
was  given  a  gain  of  either  0.8  or  1.2  before  differencing,  yielding  either  a  positive  or 
negative  mean  (and  thus  DC  component). 

The  modeling  suggests  that  if  EMD  prefilters  pass  a  significant  amount  of  DC, 
outputs  may  go  both  above  and  below  zero.  Importantly,  however,  we  see  negative-going 
transients  primarily  for  small  targets  moving  at  lower  speeds:  much  longer  targets 
produce  positive  responses  only,  as  they  cross  the  receptive  field,  independent  of  the 
model  pre-filter  that  we  select.  The  signal  arises  from  a  quasi  ‘wave  interference’  process 
whereby  the  residual  impulse  response  elicited  by  the  leading  edge  of  the  target  is 
overlapped  with  the  early  part  of  the  impulse  response  elicited  from  the  trailing  edge  of 
the  target.  A  filtering  strategy  that  is  selective  for  the  negative-going  transients  in  such 
responses  might  yield  a  huge  gain  in  response  selectivity  for  small  targets.  A  major 
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advantage  of  such  a  strategy  is  that  it  could  be  applied  even  prior  to  higher-order  small 
target  detectors  such  as  the  ESTMD  in  order  to  obtain  a  selectivity  for  events  consistent 
with  the  passage  of  a  longitudinally  small  target. 

We  developed  a  detection  algorithm  employing  this  strategy,  which  is  based  on 
taking  the  product  of  the  integral  of  the  mean  output  response  of  an  EMD,  with  the  mean 
integral  value  of  the  rectified  transient  signal  having  an  opposite  polarity.  This  algorithm 
was  shown  to  be  effective  and  unambiguous  in  simulations.  Results  of  such  simulations 
are  shown  in  Figure  16  below,  in  which  several  quantities  are  plotted  as  functions  of  the 
target  width  and  target  speed.  Panel  A  shows  the  time  averaged  (mean)  output  of  the 
‘raw’  (low-pass  pre-filtered)  EMD  array.  As  expected,  this  is  not  selective  for  small 
targets,  and  shows  sharpest  tuning  for  target  speed  when  target  size  is  smaller.  Panel  B 
shows  the  time-averaged  output  of  EMDs  after  half-wave  rectification  of  the  negative- 
going  transient  responses.  As  described  above,  these  are  most  prominent  for  small  targets 
moving  at  lower  speeds.  While  such  outputs  in  themselves  might  give  an  indication  that  a 
feature  is  a  potential  target,  it  is  important  to  note  that  this  is  inadequate  as  a  model  for 
target  selectivity  in  its  own  right,  since  targets  moving  in  the  opposite  direction  will  give 
inverted  responses  and  thus  produce  output  of  such  a  model  with  similar  tuning  to  that  in 
panel  A.  In  panel  C,  however,  we  have  ‘gated’  the  half-wave  rectified  signal  with  the 
mean  output,  which  is  in  itself  direction  selective.  This  provides  a  promising  strategy  that 
is  selective  for  longitudinally  small  image  features,  while  retaining  the  direction 
selectivity  inherent  to  EMDs. 


Target  size  (degrees)  Target  size  (degrees) 
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A)  Mean  EMD  output 


C)  Model 

trf 


10^  10^ 


Target  speed  (degrees/s) 


B)  Half-wave  rectified  (negative)  EMD  outpul 


Figure  16:  Making  use  of  the  characteristic  response  of  EMD  (with  lowpass 
prefiltering)  to  longitudinally  small  moving  targets:  a  simple  second-order  motion 
processing  model  with  selectivity  for  small  targets  and  direction  of  motion.  A)  is  the 
time-averaged  EMD  output,  B)  the  time  average  of  negative  half-wave  rectification, 
and  C)  the  average  model  output,  all  as  functions  of  target  size  and  speed.  Brighter 
colors  in  the  false-color  images  indicated  larger  values. 


Note  the  consistency  of  the  results  shown  in  Figure  16C  with  the  STMD  responses  in 
the  left  panel  of  Figure  4:  The  optimum  response  shifts  to  higher  target  speeds  as  target 
size  increases. 

Finally,  we  note  that  this  model  represents  a  particular  case  of  a  general  approach  to 
filtering  EMD  outputs  for  events  consistent  with  the  passage  of  longitudinally  small 
visual  objects:  that  is,  ‘looking  for’  features  in  the  EMD  output  that  are  particular  to  such 
events.  The  presence  of  a  triphasic  response  is  an  especially  distinctive  feature,  but  others 
might  also  be  useful.  For  example,  in  the  case  in  which  perfect  bandpass  filtering 
precedes  the  EMD,  then  a  bimodal  response  that  remains  well  above  zero  between  the 
peaks  might  be  taken  as  an  indication  of  small  target  passage.  Also,  although  we  have 
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implemented  the  processing  here  by  a  non-linear  operation  (rectification)  there  are 
alternative  and  equally  plausible  (in  the  biological  sense)  strategies  (e.g.  linear  wavelet 
filtering)  that  could  be  used  to  extract  the  same  temporal  response  ‘features’. 

3. 2.3  Final  Status  of  STMD  Modeling 

Based  on  the  results  of  the  neurobiologcal  and  modeling  efforts  undertaken  during  the 
project,  we  now  posit  the  following  major  functional  elements  for  small  target 
processing,  leading  to  wide-field  STMD  capability: 

1.  Elementary  motion  detector  (EMD):  The  correlational  EMD  or  its  constituent 
parts  (i.e.,  correlator  units)  are  still  regarded  as  the  likely  fi-ont-end  processing 
for  STMD  neurons.  Evidence  is  indirect,  but  there  have  been  no  findings  yet  to 
contradict  this  supposition. 

2.  Longitudinal  target  size  discrimination:  Ability  to  discriminate  ‘smallness’  of 
target  in  direction  of  motion.  The  relevant  modeling  reported  in  Section  3.2.2  is 
consistent  with  the  experimental  results  exemplified  by  Figure  4  in  Section  3.1.2. 
Such  processing  may  serve  to  ‘prefilter’  the  outputs  of  EMDs  for  events 
consistent  with  passage  of  a  small  object. 

3.  Target  /  background  speed  discrimination:  This  is  an  essential  capability  needed 
to  distinguish  a  moving  target  from  a  moving  cluttered  background  (which  may 
contain  small  features  indistinguishable  from  ‘targets’).  Some  initial  efforts  at 
modeling  this  capability  in  abstract  (SPICE)  simulations  were  conducted  under 
support  of  Navy  SBIR  contract  N00014-03-M-0171.  This  approach  involves  a 
comparison  of  the  traversal  times  across  pairs  of  ommatidia  of  ‘small  target 
events’  with  the  expected  traversal  times  for  moving  background  (which  might 
well  have  features  like  small  targets).  This  comparison  is  achieved  with  an  EMD- 
like  element  that  is  tunable  by  feedback  representing  state  of  local  backgroimd 
motion,  and  which  performs  anticorrelations  rather  than  correlations  so  that  it  is 
responsive  to  events  inconsistent  with  that  motion. 

4.  Lateral  target  size  discrimination:  Ability  to  discriminate  ‘smallness’  of  target 
perpendicular  to  direction  of  motion.  This  (along  with  enhancement  of  directional 
selectivity)  may  be  a  principal  function  of  cells  that  we  have  labeled  ESTMDs  as 
a  result  of  work  on  this  project. 

5.  ‘Dendritic  processing’:  A  mechanism  in  a  wide-field  collator  cell  for  stimulus 
summation  and  artifact  rejection.  Reinforcement  of  either  active  or  diffusive 
potentials  along  a  dendrite  is  presumed  to  occm  by  sequential  excitement  of 
ESTMDs  along  continuous  tracks  in  visual  space.  This  feature  exploits  the 
constraint  of  continuity  of  motion  to  obtain  higher-confidence  detection  of  a 
moving  target.  Although  not  modeled  in  simulations,  this  capability  was  modeled 
in  silicon  as  detailed  below  in  Section  3.3.4. 
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3.3  Silicon  Modeling:  Circuit  Designs  and  Results 

3.3.1  Transistor  and  Basic  Test  Structures 

The  transistor  test  structures  and  other  small  test  circuits  placed  on  the  test  chips 
described  in  Section  2.3.1  were  intended  to  allow  evaluation  of  basic  characteristics  that 
impact  larger  circuits  implementing  STMD  models  (and  in  fact  in  many  cases,  analog 
neuromorphic  designs  in  general).  Perhaps  the  most  important  of  these  is  matching  of 
device  properties  from  transistor  to  transistor,  which  affects  signal  accmacy  and  the 
ability  to  set  reference  levels  in  virtually  every  circuit  design  considered.  Also  of  interest 
were  leakage  currents  (affecting  the  magnitude  of  time  constants  obtainable  in  dynamic 
circuits),  subthreshold  behavior,  and  functionality  of  novel  devices  fabricated  in  silicon- 
on-sapphire. 

Results  from  the  Agilent  0.5pm  bulk  CMOS  process  (used  for  chip_l_bulk) 
suggested  that  it  is  very  well-controlled,  with  good  device  matching  and  extremely  low 
leakage  currents.  Current  mismatches  among  inside  pairs  of  5pm  square  devices  were 
generally  well  under  10%,  with  standard  deviations  of  the  extracted  threshold  difference 
being  less  than  3mV  for  PMOS  and  less  than  2mV  for  NMOS,  on  a  sample  size  of  eight 
chips.  Smaller  samples  were  evaluated  for  2.3pm  square  and  8pm  square  devices,  so 
these  results  are  less  definitive,  but  the  matching  of  the  larger  transistors  appears  to  be 
consistent  with  the  generic  model  (standard  deviations  varying  inversely  with  the  square 
root  of  device  area).  The  smaller  transistors,  however,  display  poorer  matching  than 
would  be  expected  from  results  on  the  larger  devices,  suggesting  that  the  lithographic 
limits  and  other  second-order  factors  may  be  influencing  their  characteristics. 

Additionally,  common  centroid  quads  with  total  device  area  of  25  pm^  were  tested  for 
comparison  with  the  individual  square  devices.  Device  matching  for  the  common 
centroid  geometry,  when  amortized  by  area  of  the  entire  layout,  is  about  20%  better  for 
n-channels  and  no  better  for  p-channels,  relative  to  square  devices.  Given  the  impact  of 
the  common  centroid  approach  on  complexity  and  routing,  we  concluded  that  simple 
matched  pairs  would  be  the  layout  technique  of  choice.  Unfortunately,  loss  of  access  to 
the  Agilent  process  precluded  further  circuit  development  in  this  process. 

Testing  of  individual  transistors  on  STMDjchip,  fabricated  in  TSMC  0.35pm 
mixed-signal  bulk  CMOS,  showed  somewhat  poorer  matching  than  for  devices  of 
comparable  size  fabricated  in  the  Agilent  0.5pm  process,  particularly  for  p-channel 
devices.  The  devices  tested  were  4.8pm  square  and  were  arranged  in  matched  inside 
pairs.  Standard  deviation  of  NMOS  threshold  variations  was  on  the  order  of  5-6mV,  but 
was  8-9mV  for  PMOS.  This  may  necessitate  the  use  of  larger  PMOS  devices  for  design, 
or  (more  likely)  doubling  up  of  transistors  (use  of  dual  pairs)  in  match-critical  circuit 
applications.  In  addition,  tests  of  dynamic  circuits  suggested  larger  leakage  currents  than 
in  the  Agilent  process,  probably  limiting  time  constants  to  less  than  Is. 

Device  matching  on  the  SOS  chips  (the  gratis  chip  and  chip_l_sos)  gave  evidence  of 
poorer  matching  than  either  bulk  process.  For  NMOS,  the  standard  deviation  of  threshold 
variations  was  found  on  the  order  of  6mV  for  5  pm  square  devices  on  the  gratis  chip,  but 
this  result  excludes  a  significant  number  of  outliers  in  the  sample  with  mismatches  larger 
than  50mV  (a  value  that  renders  them  useless  for  analog  design  purposes).  Threshold 
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mismatches  were  somewhat  smaller  as  expected  for  8pm  square  devices  on  chip_l_sos, 
but  the  statistics  were  questionable  due  to  the  continued  presence  of  outliers  with  very 
large  mismatches.  PMOS  on  chip_l_sos  (8pm  square  devices)  gave  standard  deviations 
of  about  3.5mV.  Lightly  doped  MOS  transistors,  as  expected,  gave  improved  matching 
performance,  particularly  for  NMOS  devices. 

Because  the  n-channel  transistors  in  this  process  are  particularly  susceptible  to 
floating  body  effects  (commonly  manifest  in  the  “kink  effect,”  a  reduction  in  drain 
resistance  in  saturation),  we  speculated  that  the  large  mismatches  may  actually  arise  from 
the  floating  body  rather  than  the  channel  interface.  (This  speculation  was  confirmed  by 
results  from  devices  with  body  connections  that  were  fabricated  and  evaluated  under 
support  of  another  project.  However,  layout  of  devices  with  compact  body  connections 
requires  violation  of  existing  design  rules,  and  makes  verification  difficult  if  not 
impossible.) 

Test  results  from  lateral  bipolar  junction  transistors  on  the  SOS  chips  were  generally 
negative.  Ungated  devices  showed  evidence  of  high  collector-emitter  conductivity  and 
little  transistor  action,  suggesting  that  the  source-drain  block  is  not  entirely  effective  at 
the  small  base  width  geometries  needed  for  reasonable  current  gains.  The  gated  devices 
showed  transistor  action,  although  at  higher  currents  the  slope  of  the  Vb  /  Ic  relationship 
was  considerably  smaller  than  the  inverse  thermal  voltage,  suggesting  series  resistance  or 
other  nonidealities  at  work.  Current  gains  were  reasonable  for  the  gated  npn  devices  (10- 
30  at  IpA  collector  currents,  higher  at  nA  level  currents),  but  for  the  pnp  devices  were 
considerable  poorer  (current  gains  from  less  than  unity  to  three).  Leakage  currents  at  low 
base  voltages  were  substantial  in  the  npn  devices  (not  surprising  since  the  vendor  now 
recommends  a  minimum  gate  length  of  0.8pm  for  their  regular  NMOS  devices  due  to 
leakage  concerns).  Additionally,  device  matching  among  the  BJTs  appeared  to  be  no 
better  than  that  of  MOSFETs.  Our  conclusions  were  that  lateral  BJTs  would  not  be 
generally  useful. 

Depletion  mode  devices  fabricated  in  SOS  functioned  as  expected.  No  statistics  were 
taken  for  device  matching  in  these  transistors. 

Results  on  primitive  devices  in  the  SOS  process  were  significant  in  convincing  us 
that  further  development  should  be  pursued  in  bulk  silicon. 

3.3.2  Log-Domain  Filter  Circuit 

During  the  course  of  the  project,  a  novel  methodology  for  the  implementation  of  long 
time-constant  log-domain  filter  circuits  was  devised  and  developed.  The  concept  is 
described  in  detail  in  a  manuscript  prepared  for  publication  and  previously  passed  to  the 
contracting  officer’s  representative;  we  describe  the  fundamentals  of  this  circuit  and  its 
development  here  for  completeness. 

A  schematic  diagram  of  the  basic  circuit  is  depicted  in  Figure  17; 
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Figure  17:  Basic  log-domain  filter  circuit. 

The  circuit  is  operated  in  the  subthreshold  regime,  and  takes  advantage  of  the  fact  the 
‘resistive’  element  M3  has  an  exponential  current-voltage  relationship  similar  to  the 
current  mirror  devices  Ml  and  M2.  The  input  and  output  are  both  unipolar  current 
signals.  Under  idealized  assumptions  regarding  the  transistors  in  the  circuit  (absence  of 
junction  leakage  and  body  effect,  and  drain  current  independence  of  drain  voltage  in 
saturation),  it  can  be  shown  that  the  input-ou^ut  relation  may  be  written: 

(l  +  CU.r/loRd/dt)Io=Ii, 

where  the  input  and  output  transistors  Ml  and  M2  are  assumed  matched,  Iqr  is  the 
specific  ciurent  parameter  for  M3,  and  where  Ut  represents  the  thermal  voltage,  h  and  U 
represent  the  input  and  output  currents  (the  drain  currents  of  Ml  and  M2,  respectively). 
The  circuit  thus  implements  a  linear  first-order  lowpass  filter  (a  real  pole)  in  the  current 
domain,  with  time  constant  x  =  CU.j.  /  Iqr  • 

Two  of  the  assumptions  made  regarding  the  idealized  circuit  are  effectively  true  when 
it  is  fabricated  in  silicon-on-sapphire:  there  is  no  junction  leakage,  due  to  dielectric 
isolation  of  the  individual  devices,  and  although  the  body  effect  is  present,  it  does  not 
significantly  affect  external  linearity  of  the  circuit,  due  to  the  ‘floating  substrates’  of  the 
transistors.  Both  of  these  assumptions,  however,  are  not  valid  for  bulk  silicon  and  have 
significant  implications  for  circuit  performance.  The  most  serious  of  these  is  dependence 
of  the  time  constant  on  mean  current  levels.  The  third  assumption,  independence  of  drain 
current  from  drain  voltage,  can  be  made  approximately  tme  in  either  SOS  or  bulk  silicon 
by  the  use  of  long-channel  devices. 

Simple  modifications  to  this  circuit  allow  electrical  controllability  of  its  time 
constant.  These  involve  level-shifting  the  gate  of  the  device  acting  as  a  nonlinear  resistor 
(M3  in  Figure  17)  with  a  follower  transistor  as  illustrated  in  Figure  18  below: 
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Figure  18:  First-order  lowpass  filters  with  level-shifting  followers  to  modify  time 
constant.  In  (a)  the  gate  voltage  of  the  nonlinear  resistor  device  is  shifted  by  a 
follower  transistor  of  the  opposite  type.  In  (b),  it  is  shifted  by  connecting  it  to  the 
gate  of  a  follower/buffer  of  the  same  type. 

In  the  circuits  of  Figure  18,  the  level-shift  voltage,  and  thus  the  time  constant,  is 
controlled  via  the  follower  bias  current  Is.  The  operation  of  these  circuits  is,  however, 
restricted  by  bandwidth  requirements  on  the  follower  circuits  in  them. 

One  reason  that  this  circuit  is  of  interest  for  this  project  is  that  it  provides  a  means  to 
model  the  nonlinear,  rapid-activation,  slow-decay  characteristics  that  are  reflected  in  the 
‘tonic’  ESTMD  response.  When  input  and  output  current  levels  are  increased  from  weak 
inversion  to  moderate  or  strong  inversion  levels,  then  the  charging  and  discharging 
behavior  of  the  filter  becomes  asymmetric.  When  operated  in  strong  inversion,  it  can  be 
shown  that  the  nonlinear  filter  transfer  characteristic  takes  the  form  (using  the  Heaviside 
notation) 


In^ 

1 

2  J 

(l  +  zd !  dt) 

? 

where  Iin  and  Iqut  are  the  input  and  output  currents,  respectively,  K  is  the  transfer 
constant  (in  AA^^)  of  the  input  and  output  transistors,  t  is  the  time  constant  of  the  filter, 
and  Ut  is  the  thermal  voltage.  (This  assumes  that  the  transistor  serving  as  a  nonlinear 
resistor  in  the  circuit  remains  in  weak  inversion,  as  would  be  the  case  for  all  but 
extremely  large  and  rapid  excursions  of  the  input).  Simulated  temporal  response  of  the 
filter  to  a  square  wave  input  when  operated  in  the  moderate  to  strong  inversion  regions  is 
depicted  below  in  Figure  19: 


Log  Filter  (moderate  to  stong  inversion) 


Time  (ms) 

Figure  19;  Log  domain  filter  response  when  operated  in  moderate  to  strong 

inversion. 

Similar  nonlinear  characteristics,  with  even  more  pronounced  asymmetry  between  charge 
and  discharge,  can  be  obtained  with  cascoded  circuits  to  be  discussed  below. 

The  basic  circuit  of  Figure  17,  when  fabricated  in  SOS,  gave  generally  positive 
results.  The  n-channel  version  showed  some  nonlinearity  in  subthreshold  (pseudolinear) 
operation,  but  the  linearity  of  the  p-channel  version  was  good  (total  harmonic  distortion 
on  the  order  of  3%  was  measured  at  the  -3dB  point  in  experiments  with  sinusoidal 
inputs).  Natural  time  constants  were  on  the  order  of  hundreds  of  milliseconds  (reflecting 
relatively  high  channel  leakage  in  the  SOS  devices).  In  the  version  of  the  circuit  with 
electrical  control  of  the  time  constant,  the  use  of  a  low-doped  n-channel  follower 
permitted  a  practical  variation  of  that  time  constant  over  a  five-decade  range. 

When  the  filter  circuit  was  operated  with  the  input  and  output  devices  in  moderate  to 
strong  inversion,  it  displayed  the  expected  rapid  charge  /  slow  discharge  characteristic. 
Figure  20  below  shows  an  example  of  data  obtained  fi:om  a  p-channel  SOS  circuit 
(version  with  electrical  time  constant  control),  in  response  to  a  square  wave  input.  Due  to 
the  polarity  of  the  circuit  and  the  buffer  amplifiers  used  for  interface,  increments  are 
represented  by  negative  excursions  and  decrements  by  positive  excursions  in  the  input 
and  output  traces.  The  circuit  time  constant  was  set  arbitrarily  to  shovv  the  disparity 
between  charging  and  discharging,  at  the  given  input  firequency  (500Hz). 
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Tek  Run:  I00ks/s  Sample 


Figure  20:  Operation  of  the  p-channel  SOS  log  domain  filter  in  moderate  to  strong 
inversion.  Top  (trace  1)  is  the  square  wave  input  and  bottom  (trace  2)  is  the  circuit 
output.  The  effective  vertical  scale  is  O.SpA  per  division.  Increasing  current  is  in  the 
downward  direction  for  the  ordinate. 

When  fabricated  in  the  Agilent  0.5)j,m  bulk  CMOS  process,  the  basic  log-domain 
filter  circuit  (both  NMOS  and  PMOS  versions)  was  untestable  due  to  the  magnitude  of 
the  time  constants,  reflecting  extremely  low  channel  and  jimction  leakage  currents  in  this 
process.  Only  an  NMOS  version  with  a  level-shifting  follower  proved  testable,  and  it 
showed  good  performance  and  linearity,  although  comer  frequencies  higher  than  a  few 
hundred  mHz  could  not  be  obtained.  However,  it  also  showed  evidence  of  the  significant 
negative  impact  of  the  body  effect  in  bulk  silicon:  dependence  of  the  circuit  time  constant 
on  mean  filter  state.  When  the  circuit  is  operated  in  the  nonlinear  ‘tonic’  regime,  this 
results  in  less  difference  between  the  fast  time  course  of  excitation  and  the  slow  post- 
excitatory  discharge,  than  in  the  same  circuit  without  the  body  effect  (e.g.,  as 
implemented  in  SOS),  in  addition  to  rendering  the  temporal  characteristics  dependent  on 
the  mean  input  current. 

A  solution  to  this  problem  was  developed  under  joint  support  of  this  contract  and  US 
Air  Force  SBIR  Contract  F08630-02-C-0013.  By  using  a  follower  transistor  of  the  same 
t5q)e  as  the  filter  for  electrical  control  of  the  time  constant,  the  body  effect  in  the  follower 
can  be  used  to  cancel  the  body  effect  in  the  main  filter  itself.  This  configuration  is  made 
possible  by  placing  the  follower  in  a  separate  well,  which  is  biased  below  the  positive 
supply,  in  order  to  promote  the  follower  circuit  bandwidth  well  beyond  that  of  the  filter 
proper. 

In  addition,  a  new  approach  to  obtaining  the  ‘tonic’  nonlinear  behavior  was 
developed.  The  ‘tonic’  property  was  originally  obtained  by  applying  above-threshold 
currents,  resulting  in  a  sub-exponential  current-voltage  characteristic  in  the  input  and 
output  devices,  and  breaking  the  pseudolinearity  of  the  circuit.  This  has  the  disadvantage 
of  requiring  at  least  microampere-level  currents  at  every  cell  in  an  array.  However,  the 
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same  effect  can  be  obtained  by  using  cascode  devices  to  force  a  sub-exponential  current- 
voltage  characteristic.  This  allows  the  circuit  to  be  operated  in  the  tonic  mode  even  in  the 
subthreshold  regime,  and  converts  it  back  to  a  micro-power  circuit. 

The  filter  design  used  on  the  final  chip,  fabricated  in  the  TSMC  0.35pm  bulk  CMOS 
process,  is  depicted  in  schematic  form  in  Figure  21  below: 


Figure  21:  “Tonic’  filter  implementation.  The  VPP  port  represents  an  additional 
power  supply  terminal  that  is  supposed  to  be  held  at  a  potential  at  least  several 
hundred  millivolts  below  the  positive  supply  (VDD).  The  transistor  labeled 
‘follower’  is  situated  in  a  separate  n-well  that  is  maintained  at  VPP  rather  than 
VDD.  The  transistors  with  gates  connected  to  CASCODE,  when  properly  biased, 
force  a  sub-exponential  current-voltage  characteristic  on  the  input  and  output 
stages. 

Functional  tests  showed  that  this  circuit  works  well  in  both  the  pseudolinear  and  ‘tonic’ 
regimes.  Data  were  included  in  the  manuscript  submitted  for  publication. 

3.3.3  ESTMD  Model  Arrays 

Analogs  of  ESTMDs  were  included  on  all  test  chips  fabricated  during  the  project. 
The  general  architecture  of  the  unit  STMD  cell  is  illustrated  (in  this  case,  for  the  final 
version  implemented  in  bulk  silicon  on  ESTMD_chip)  in  Figure  22: 
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Figure  22:  Architecture  of  the  unit  ESTMD  cell  on  ESTMD  chip.  VPP  is  the 
independent  well  bias  discussed  in  the  text.  The  bias  applied  at  TNBIAS  determines 
the  time  constant  of  the  ‘tonic’  filter,  and  the  CASCODE  bias  is  used  to  force  a 
subexponential  characteristic  on  the  input  and  output  of  the  filter.  The  potential 
difference  between  GBIAS  and  SBIAS  determines  the  space  constant  for  the  spread 
of  diffusive  inhibitory  signals.  All  biases  are  global  in  an  array  of  ESTMDs.  In  this 
circuit,  direction  4  (downward  in  the  array)  is  the  preferred  direction,  due  to  the 
absence  of  a  directed  inhibitory  signal  in  that  direction. 


ESTMD  unit  cells  were  arranged  in  hexagonal  arrays  in  test  structures  on  each  of  the 
fabrcated  test  chips.  Only  local  interconnections  to  six  nearest  neighbors  are  made  by 
eaeh  cell  in  this  array.  A  diffusive  network  is  used  to  propagate  inhibition.  In  each  array, 
the  central  37  cells  were  addressable  one  at  a  time  for  input,  and  the  output  of  the  single 
cell  at  the  center  was  brought  out  for  observation.  The  array  was  stimulated  during 
testing  using  a  “single  pixel”  stimulus  without  directional  properties,  and  of  a  fixed 
amplitude,  which  was  steered  across  the  array  in  a  particular  direction  by  applying  a 
sequence  of  addresses.  Thus  the  model  represents  a  ‘spatially  compressed’  version  of  a 
biological  ESTMD,  with  input  preprocessing  resulting  in  a  stereotyical  pulse  input  in 
response  to  a  ‘small  moving  target  event.’ 

In  these  arrays,  the  relative  scaling  of  inhibition  and  excitation  is  important  in  order 
to  obtain  a  net  excitatory  response  to  stimuli  moving  in  the  preferred  direction,  and  net 
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inhibition  in  other  directions.  This  scaling  is  accomplished  by  the  sizing  of  transistors  in 
the  circuit.  In  our  initial  design  efforts,  full  transistor-level  simulations  of  the  array  were 
used  to  determine  appropriate  device  sizing,  but  these  simulations  proved  problematic 
with  regard  to  convergence.  Ultimately,  some  results  were  obtained  and  used  to 
determine  relative  scaling  of  inhibitory  and  excitatory  signals  in  the  SOS  implementation 
on  chip_l_sos,  although  there  was  evidence  of  numerical  problems.  For  subsequent 
design  efforts  (for  chip_ljbulk  and  ESTMDjchip)  we  developed  a  version  of  the  array 
with  abstract  elements,  which  still  retained  the  first-order  characteristics  of  the  circuit 
model,  but  for  which  convergence  was  improved. 

Test  results  on  the  SOS  array  validated  the  general  principle  of  the  ESTMD  design, 
although  improper  current  scaling  was  evident.  Figure  23  shows  the  time-course  of  the 
output  of  the  central  EMD  cell  in  the  array,  for  five  different  ‘stimulus  paths’  across  its 
receptive  field.  These  are  arranged  in  the  order  of  the  level  of  excitation  they  elicit,  from 
the  initial  path,  which  does  not  cross  the  excitatory  region  at  all,  to  the  final  path,  which 
crosses  the  excitatory  region  in  the  preferred  direction.  In  this  particular  design,  the 
polarities  are  inverted  relative  to  neuron  membrane  potentials,  with  inhibition  represented 
by  positive-going  voltages  and  excitation  by  negative-going  voltages.  The  input  current 
was  switched  from  cell  to  cell  each  50ms,  which  would  correspond  to  a  ‘speed’  of  30 
degrees  per  second  if  the  ‘cells’  were  associated  with  ommatidia  with  a  separation  angle 
of  1.5°.  The  time  constant  in  the  nonlinear  filter  circuit  was  set  by  hand  so  that  signal 
decay  is  relatively  slow  compared  to  the  50ms  transition  time  (whereas  the  onset  is  rapid 
enough  to  equilibrate  in  less  than  that  time). 


Figure  23:  Results  from  the  SOS  ESTMD  array.  The  icon  at  upper  left  shows  the  different  paths 
across  the  ESTMD  receptive  field  of  the  central  cell,  over  which  an  input  current  pulse  (a 
“moving  target”)  was  stepped.  The  excitatory  center  in  this  icon  is  depicted  in  red,  with  the 
preferred  direction  downward.  Each  trace  shows  the  central  ESTMD  output  for  one  of  the  input 
paths.  Effective  vertical  scale  is  IpA  per  division.  Inhibition  is  represented  by  positive  voltages 
and  excitation  by  negative,  ‘Resting  potential’  is  two  divisions  up  from  the  bottom  of  each 
graph. 
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In  the  results  in  Figure  23,  the  excitation  when  the  input  crosses  the  receptive  field 
center  is  not  sufficient  to  overcome  the  effects  of  inhibition,  even  in  the  preferred 
direction.  However,  the  results  are  qualitatively  as  desired,  as  the  closest  approach  to  net 
excitation  does  occur  in- the  preferred-direction  case  (Figure  23e). 

Interestingly,  when  new  simulations  of  this  circuit  were  run  with  abstract  components 
replacing  the  active  devices,  the  scaling  mismatch  observed  between  the  excitatory  and 
inhibitory  currents  was  the  opposite  of  what  was  seen  experimentally:  the  simulated 
excitatory  currents  were  too  large  relative  to  the  inhibitory  component,  rather  than  the 
other  way  around.  This  is  illustrated  below  in  simulation  results  depicted  in  Figure  24.  In 
these  results,  both  preferred  and  antipreferred-direction  stimuli  resulted  in  net  excitation 
(positive  polarity  in  this  case).  Repeated  examination  and  analyses  of  both  the  design  and 
the  simulated  circuit  failed  to  uncover  any  reason  for  this  discrepancy:  component  scaling 
in  both  was  found  to  be  as  assumed,  and  these  two  scalings  matched.  We  concluded  that 
there  must  be  a  systematic  but  non-design-related  cause  to  the  scaling  problem.  It  might 
be  due  to  systematic  mismatch  in  transistor  properties  in  the  layout  (caused,  for  example 
by  transistor  positions  relative  to  other  devices),  or  there  might  be  another  cause,  such  as 
a  deviation  from  expected  threshold  that  forces  the  excitatory  transistors  to  operate  in  the 
triode  mode  and  supply  less  than  the  expected  amount  of  current.  Either  way,  the 
discrepancy  between  simulated  and  actual  behavior  did  not  bode  well  for  design  efforts 
in  SOS,  and  it  became  another  significant  factor  in  om  ultimate  decision  to  pursue 
subsequent  designs  in  bulk  silicon  rather  than  SOS. 


Abstract  ESTMD  Array 


Figure  24:  Simulated  output  of  the  SOS  ESTMD  array,  with  devices  represented  by 
abstract  components.  Output  polarities  are  reversed  with  respect  to  the  actual 
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circuit  of  Figure  23.  The  excitatory  signal  scaling  is  too  large  in  this  case  for  it  to  be 
overcome  by  inhibition  when  motion  is  in  the  antipreferred  direction. 

In  the  ESTMD  array  on  chip_l_bulk,  the  scaling  issue  received  carefiil  attention  in 
design  and  simulation.  Unfortunately,  the  version  of  the  tonic  filter  included  in  the 
ESTMD  cell  on  this  chip  had  such  long  intrinsic  time  constants  that  it  was  not  operable  at 
practical  frequencies  on  this  chip,  and  testing  of  the  ESTMD  array  on  this  chip  proved  to 
be  infeasible. 

The  ESTMD  cell  used  in  the  final  test  chip,  ESTMD_chip,  included  the  redesigned 
version  of  the  tonic  filter  depicted  in  Figure  21.  This  allows  a  ‘tonic’  response  in  the 
subthreshold  regime,  as  well  as  controllability  of  the  time  constant.  Nominal  excitatory 
and  inhibitory  scaling  values,  as  determined  by  simulation  and  analysis,  were  used  in  the 
primary  test  structure,  under  the  assumption  that  the  test  results  obtained  from  the  SOS 
version  of  the  circuit  were  anomalous.  However,  as  a  hedge,  a  second  version  of  the 
ESTMD  array  was  included  in  which  the  scaling  of  the  excitatory  signals  was  increased, 
so  as  to  permit  the  circuit  to  function  if  in  some  way  the  required  excitatory  scaling  was 
underestimated. 

The  ESTMD  arrays  on  ESTMD_chip  were  be  set  up  so  that  a  cluster  of  seven 
adjacent  ESTMD  cells  could  be  addressed  at  once,  and  supplied  with  seven  independent 
inputs.  This  architecture  allows  the  imposition  of  input  sequences  whose  spatiotemporal 
signatures  cover  more  than  one  pixel  simultaneously.  It  was  implemented  to  allow 
simulation  of  moving  targets  of  sizes  larger  than  a  single  pixel.  In  addition,  some 
preprocessing  schemes  (such  as  the  elementary  motion  detector)  may  result  in  signatures 
that  are  more  than  one  pixel  in  spatial  extent,  even  in  response  to  single-pixel  moving 
targets.  This  array  will  be  implemented  with  a  single  preferred  direction,  as  on  the  first 
pair  of  chips. 

Preliminary  results  from  the  array  with  nominal  current  scaling  were  positive.  In 
experiments  using  a  single-pixel  pulse  stimulus  moved  across  the  array  at  a  fixed  speed 
(as  in  the  prior  chips),  the  ‘tonic’  filters  could  be  tuned  so  that  ‘excitation’  resulted  when 
the  stimulus  moved  in  the  preferred  direction,  but  not  when  the  stimulus  moved  in  other 
directions.  A  set  of  example  outputs  is  depicted  below  in  Figure  25: 
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Figure  25:  (Previous  page)  Results  from  the  bulk  silicon  ESTMD  array  on 
STMD_chip.  The  icon  at  upper  left  shows  the  different  paths  across  the  ESTMD 
receptive  field  of  the  central  cell,  over  which  an  input  current  pulse  (a  “moving 
target”)  was  stepped.  The  excitatory  center  in  this  icon  is  depicted  in  red,  with  the 
preferred  direction  downward.  Each  trace  shows  the  central  ESTMD  output  for  one 
of  the  input  paths,  as  a  function  of  time.  Inhibition  is  represented  by  negative 
voltages  and  excitation  by  positive.  ‘Resting  potential’  is  two  divisions  down  from 
the  top  of  each  graph.  The  circuit  was  operated  in  weak  inversion. 


In  the  responses  in  Figure  7,  only  stimulus  motion  in  the  preferred  direction  yielded 
an  excitatory  (net  positive)  response  when  the  stimulus  crossed  the  center  of  the  receptive 
field.  Although  this  is  as  desired,  the  response  is  weak  in  magnitude  and  there  is  little 
margin  between  its  peak  and  the  peaks  induced  when  the  stimulus  crosses  the  excitatory 
region  in  the  non-preferred  directions. 

In  an  effort  to  improve  the  performance  of  the  circuit,  we  also  ran  experiments  in 
which  it  was  operated  in  moderate  inversion.  Although  this  mode  results  in  better  noise 
margin,  the  difference  between  preferred-  and  non-preferred-direction  responses  remains 
modest  relative  to  overall  signal  levels. 


Tek  SSnS  1  OOkS/s  6  Acqs 


Figure  26:  Response  of  bulk  silicon  ESTMD  array  on  STMD  chip  (version  with 
nominal  scaling  of  excitation  and  inhibition)  as  a  function  of  time,  to  antipreferred 
(a)  and  preferred  (b)  direction  ‘small  moving  target’  stimuli.  The  circuit  is  operated 
in  moderate  inversion. 


When  the  second  ESTMD  array  with  increased  scaling  of  excitatory  current 
components  was  tuned  so  that  preferred-direction  stimuli  resulted  in  net  excitation,  then 
stimuli  crossing  the  excitatory  center  in  any  other  direction  also  caused  an  excitatory 
response,  demonstrating  that  the  excitatory  scaling  is  too  large  and  confirming  the 
suitability  of  scaling  in  the  nominal  array. 

However,  we  investigated  a  mode  of  operation  of  the  tonic  filter  that  ultimately 
allowed  the  second  array  to  function  as  desired.  When  the  input  of  the  filter  is 
overdriven,  that  is,  subject  to  voltages  at  which  its  current  levels  are  saturated,  then  the 
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output  enters  a  regime  in  which  a  nearly  constant  output  persists  after  the  end  of  the 
stimulus,  until  the  gate  voltage  of  the  output  device  discharges  below  the  saturation  level. 
This  is  illustrated  in  Figure  27  below,  in  which  the  left  panel  demonstrates  the  step 
response  of  the  filter  in  the  normal  ‘tonic’  mode,  and  the  right  panel,  in  the  saturated 
mode: 


Figure  27:  Step  response  of  the  cascaded  subthreshold  ‘tonic’  filter  in  the  ‘normal’ 
tonic  mode,  (a),  and  in  the  saturated  mode,  (b). 

When  the  filters  in  the  second  array  are  operated  in  the  saturated  mode  and  properly 
tuned,  the  persistence  of  inhibition  at  relatively  high  levels  in  the  network  allows 
supression  of  excitation  by  non-preferred-direction  stimuli,  while  still  permitting  net 
excitation  by  preferred-direction  stimuli.  This  is  illustrated  in  Figure  28  below. 
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Figure  28:  Response  of  bulk  silicon  ESTMD  array  on  STMD  chip  (version  with 
increased  excitatory  scaling)  as  a  function  of  time,  to  antipreferred  (a)  and 
preferred  (b)  direction  ‘small  target’  stimuli.  The  tonic  filter  is  operated  in 
saturated  mode. 

In  the  experiments  on  the  ESTMD  arrays,  particularly  when  operated  in  subthreshold, 
we  found  these  circuits  to  be  subject  to  low-level,  but  long-lasting,  residual  ‘inhibition,’ 
which  depresses  the  ESTMD  output  for  relatively  long  periods  after  a  stimulus  event.  On 
analysis,  we  attribute  this  to  a  design  oversight:  when  the  input  to  an  ESTMD  transitions 
from  an  addressed  state  to  an  unaddressed  state,  its  input  floats,  and  although  the  tonic 
filter  discharges  according  to  the  desired  time  history  at  first,  the  discharge  eventually 
slows  due  to  the  fact  that  the  input  transistor  enters  a  high-impedance  state  and  there  is  no 
low-impedance  path  for  current  leaving  the  integrating  node  of  the  filter.  The  filter 
continues  to  source  small  amounts  of  current  onto  the  diffusive  inhibitory  network.  This 
problem  could  be  remedied  by  using  a  weakly-biased  transistor  to  ‘shut  ofF  the  ESTMD 
input  device  when  there  is  no  externally-applied  input. 

All  of  the  results  presented  above  were  for  single  ‘pixel’  small  targets  moving  at  a 
particular  speed  (i.e.,  with  a  50ms  transition  time  per  ‘ommatidium’).  When  this  speed 
was  varied  while  retaining  a  fixed  tuning  of  the  STMD  cells,  we  found  that  the  desired 
response  (net  excitation  in  the  preferred  direction  and  that  direction  only)  could  be 
maintained  for  just  a  small  range  of  speeds  —  varying  by  a  factor  of  about  two  for  the 
nominal  array,  and  two  to  three  for  the  second  array  when  operated  with  tonic  filters  in 
‘saturated’  mode.  Because  even  in  the  ‘normal’  tonic  mode,  the  filter  decay  (i.e.,  fall 
time)  is  on  the  order  of  ten  times  faster  than  the  rise  time,  we  would  expect  a  broader 
range  of  speeds  to  be  achievable.  We  believe  that  several  factors  negatively  impact  this 
speed  tuning:  the  presence  of  persistent  inhibition;  non-optimality  of  the  relative  scaling 
of  inhibition  and  excitation  (the  nominal  array,  while  functional,  should  probably  have 
somewhat  greater  excitation);  and  finally,  the  chosen  architecture  itself  While  the  use  of 
only  nearest-neighbor  coimections  significantly  simplifies  the  physical  layout  of  the 
ESTMD  circuits,  permitting  a  broader  range  of  interconnections  would  allow 
implementation  of  ESTMD  models  with  better  discrimination  between  preferred  and 
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non-preferred  direction  stimuli,  and  broader  velocity  tuning  as  well.  The  simplest  such 
elaboration  may  be  to  allow  directed  inhibition  between  both  nearest  and  next-nearest 
neighbors. 

Finally,  the  ESTMD  arrays  were  tested  using  a  moving  stimulus  consisting  of  three 
adjacent  active  ‘pixels’  rather  than  a  single  pixel  stimulus.  This  stimulus  was  moved  in 
the  preferred  direction,  but  it  is  broader  than  the  effective  ‘notch’  in  the  inhibitory 
receptive  field.  Accordingly,  strong  inhibition  with  no  period  of  net  excitation  was 
observed  during  target  transition,  demonstrating  the  small-target  selectivity  of  the  model. 

3.3.4  Collator  Dendrite  Model 

In  a  second  and  larger  set  of  arrays  on  ESTMD_chip,  we  implemented  ESTMD 
arrays  with  the  single-pixel  input  architecture  as  on  the  initial  chips,  but  with  two 
preferred  directions,  corresponding  to  a  set  of  ESTMDs  aligned  with  two  of  the  principal 
axes  of  the  hexagonal  array.  In  addition,  we  implemented  analogs  of  dendritic  processing 
that  we  assume  may  be  present  in  collator  cells  corresponding  to  wide-field  STMD 
neurons.  This  collation  was  not  a  simple  summation.  Rather,  the  (biologically  plausible) 
approach  we  took  consisted  of  circuits  whose  response  is  enhanced  by  the  sequential 
excitement  of  ESTMDs  along  continuous  tracks.  Continuity  of  motion  is  an  expected 
constraint  for  natural  targets,  and  exploitation  of  this  constraint  could  lead  to  higher- 
confidence  detection  of  target  motion. 

The  basic  concept  behind  this  approach  is  illustrated  in  the  schematic  diagram  of 
Figure  29  below: 


E(Ha+)  E(Na+)  E(Wa+)  E(TJa+)  E(Wa+) 


Figure  29:  Dendrite  anaiog.  Membrane  properties  approximated  by  iumped 
parameters.  Sequential  activation  of  input  synapses  reinforces  a  diffusive  wave  of 
excitation  in  the  dendrite. 


The  continuous  membrane  capacitance,  resistance,  and  cytoplasmic  resistances  were 
modeled  with  lumped-parameter  elements.  Synaptic  inputs  were  assumed  to  be  firom 
(retinotopically-organized)  ESTMDs,  and  were  modeled  as  excitatory  and  unipolar,  i.e., 
they  are  active  only  when  the  corresponding  ESTMD  receives  a  spatiotemporal  input  that 
results  in  its  excitation. 

In  the  dendritic  analogs  of  the  STMD  circuit,  the  first-order  lowpass  characteristic 
associated  with  each  RC  stage  in  Figure  29  was  implemented  with  an  active  log-domain 
filter  circuit,  analogous  to  that  in  the  ESTMDs.  If  the  cascode  transistors  in  this  filter 
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circuit  are  turned  full  “on,”  and  their  input  currents  from  the  ESTMDs  are  at  weak 
inversion  levels,  then  they  will  act  as  linear  filters,  and  signal  propagation  in  the 
‘dendrite’  will  model  a  set  of  superposed  diffusive  responses.  However,  by  adjusting  the 
cascode  bias,  they  may  be  operated  in  the  same  nonlinear  ‘tonic’  mode  as  the  circuits  in 
the  ESTMDs  -  fast  excitation,  slow  decay  of  excitation  -  in  order  to  mimic  active 
membrane  properties  that  might  result  in  tonic  behavior.  This  could  be  used  to  strengthen 
the  reinforcement  of  a  ‘wave’  of  excitation.  Naturally,  we  expedt  the  temporal  tuning  of 
the  filters  to  be  reflected  in  the  velocity  tuning  of  the  ‘dendrite,’  and  presumably  it  should 
be  at  least  somewhat  matched  to  the  velocity  tuning  of  the  ESTMD  cells  at  the  front  end. 

In  the  STMD  circuit  appearing  on  the  final  test  chip,  individual  ‘dendritic’  segments 
have  retinotopic  ‘receptive  fields’  four  stages  long  in  the  direction  of  motion  sensitivity, 
and  two  ESTMDs  wide.  The  wide-field  STMD  cells  sum  the  outputs  of  ‘dendrites’  at 
several  overlapping  locations  in  the  retinotopic  array,  and  aligned  with  two  principal 
directions  of  the  hexagonal  lattice,  resulting  in  a  somewhat  broader  direction  selectivity 
(on  the  order  of  60°)  than  the  individual  ESTMDs  supplying  its  inputs. 

Testing  and  diagnosis  of  the  dendritic  analog  circuits  was  limited  due  to  exhaustion 
of  time  and  funds  on  the  project.  However,  we  present  some  representative  results  of  the 
limited  tests  performed,  below  in  Figure  30.  In  these  experiments,  the  ‘small  target’ 
stimulus  was  moved  across  the  array  in  several  directions,  and  the  dendrite  filter  stages 
were  biased  the  same  as  the  ESTMD  filters  (i.e.,  in  ‘tonic’  mode). 


Figure  30:  Response  of  an  STMD  ‘dendrite’  circuit  to  ‘target’  motion  in  its 
preferred  (a)  direction  and  a  non-preferred  (b)  direction,  as  a  function  of  time.  In 
both  cases,  an  artifactual  initial  excitation  is  present,  followed  by  building  excitation 
in  the  preferred  direction,  but  only  some  small  excitatory  spikes  in  the  non¬ 
preferred  direction. 


In  these  results,  for  as-yet  undiagnosed  reasons,  there  is  a  burst  of  excitation  at  the 
assertion  of  the  first  one  or  two  addresses  in  the  sequence  used  to  steer  the  ‘moving 
target’  stimulus  across  the  array  (this  artifact  may  be  due  to  improper  boundary 
conditions  on  the  ESTMD  cells  in  the  array,  leading  to  fiill  excitation  of  the  ‘dendrite’ 
containing  these  cells).  Following  this,  excitation  rebuilds  in  the  case  of  the  preferred 
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direction  stimulus,  dying  away  slowly  after  the  end  of  the  ‘motion,’  but  in  the  non¬ 
preferred  case,  there  are  only  a  few  ‘breakthrough  spikes’  of  excitation,  presumably  due 
to  ESTMD  outputs  just  momentarily  reaching  the  threshold  of  excitation. 

3.4  SUMMARY  AND  CONCLUSIONS 

In  the  neurobiological  portion  of  our  work,  we  characterized  the  responses  of  a  range 
of  neurons  that  can  be  considered  small  target  movement  detectors;  developed 
preliminary  classifications  for  some  of  these  cells,  including  a  small-field  STMD  that  we 
have  labeled  an  ‘elementary  small  target  movement  detector’,  or  ESTMD,  and  which 
may  be  an  early  or  intermediate  stage  in  a  hierarchy  of  STMD  processing;  discovered 
that  at  least  some  STMD  neurons  are  capable  of  responding  selectively  to  small  moving 
targets  against  moving  cluttered  backgrounds;  and  obtained  data  from  an  STMD  neuron 
that  we  believe  will  prove  identifiable  from  individual  to  individual  of  a  particular 
species  (E.  tenax). 

In  our  modeling  work,  we  developed  conceptual  and  numerical  models  for  the  small- 
field  STMD  neurons  that  we  have  labeled  ESTMDs;  studied  the  responses  of  elementary 
motion  detectors  (the  putative  front-end  for  STMD  neurons)  to  small  target  motion; 
developed  a  model  for  processing  that  could  endow  STMD  neurons  with  selectivity  for 
small  target  size  in  the  direction  of  motion;  examined  possible  mechanisms  for  collation 
of  ESTMD  outputs;  and  developed  a  revised  hierarchy  of  STMD  processing  in  view  of 
the  discovery  of  neurons  capable  of  distinguishing  moving  targets  against  moving 
backgrounds. 

In  our  effort  to  implement  neuromorphic  analog  circuit  models  of  STMDs,  we 
evaluated  primitive  devices  in  two  different  CMOS  processes  (standard  bulk  silicon 
mixed-signal  CMOS  and  silicon-on-sapphire);  invented  a  novel  log-domain  filter  circuit 
that  can  be  operated  in  a  nonlinear  mode  to  mimic  the  ‘tonic’  property  of  ESTMD 
neurons;  implemented,  tested  and  demonstrated  silicon  models  for  the  ESTMD;  and 
implemented  and  performed  pilot  testing  of  a  model  for  dendritic  collation  in  a  wide-field 
STMD  analog. 


4  PERSONNEL  SUPPORTED 

Patrick  Shoemaker,  Tanner  Research,  Inc.,  Co-Investigator 

David  O’Carroll,  University  of  Adelaide,  Co-Investigator 

Roger  DuBois,  University  of  Adelaide,  Post-Doctoral  Fellow 

Tamath  Rainsford,  University  of  Adelaide,  Post-Doctoral  Fellow  (partial  support) 

Andrew  Straw,  University  of  Adelaide,  Graduate  Student  (limited  support) 

Irene  Moyer,  University  of  Adelaide,  Undergraduate  vacation  scholar 
Thomas  Bartolac,  Tanner  Research,  Inc.,  Senior  Scientist  (limited  support) 


5  PUBLICATIONS 


Contract  F49620-01-C-0030  Final  Report 


50 


5.1  Publications  Wholly  or  Partly  Supported  by  this  Contract 

Shoemaker,  P.  A.  (2003)  A  methodology  for  long  time  constant  log-domain  filters  in 
CMOS.  Submitted  to  Analog  Integrated  Circuits  and  Signal  Processing. 

DuBois,  R.A.,  O’Carroll,  D.,  and  Shoemaker,  P.A.  (2003)  Spatio-temporal  tuning  for 
small  targets  from  a  simulated  array  of  elementary  motion  detectors,  in  N.  Eisner  &  H. 
Zimmermann,  Eds.,  The  Neurosciences  —  From  Basic  Research  to  Therapy:  Proceedings 
of  the  29th  Gottingen  Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag, 
Stuttgart,  pp.  547-548. 

Shoemaker,  P.  A.,  O’Carroll,  D.  C.,  and  Straw,  A.  D.  (2001)  Implementation  of  visual 
motion  detection  with  contrast  adaptation.  Proceedings  of  SPIE,  Electronics  and 
Structures  for  MEMS  (Adelaide,  December  2001),  Vol.  4591,  pp.  316-327. 

O’Carroll,  D.C.  (2001)  Biomimetic  visual  detection  based  on  insect  neurobiology. 
Proceedings  of  SPIE,  Electronics  and  Structures  for  MEMS  (Adelaide,  December  2001), 
Vol.  4591,  pp.  1-10. 

O’Carroll,  D.C.,  and  Henry,  J.  (2001)  Visual  detection  of  prey  by  dragonflies.  Presented 
at  the  International  Conference  for  Invertebrate  Vision  (Backaskog  Castle,  Sweden, 
August  7-12). 

5.2  Publications  in  Related  Fields  by  the  Principals 

O’Carroll,  D.C.,  Straw,  A.D.,  Shoemaker,  P.A.,  and  Rainsford,  T.  (2003)  Adaptive  gain 
control  in  insect  motion  detection,  in  N.  Eisner  &  H.  Zimmermann,  Eds.,  The 
Neurosciences  —  From  Basic  Research  to  Therapy:  Proceedings  of  the  29th  Gottingen 
Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag,  Stuttgart,  pp.  546-547. 

Straw,  A.D.  &  O’Carroll,  D.C.  (2003)  Ghosting  and  Aliasing  Artifacts  in  Apparent 
Motion  Displays  Eliminated  with  Motion  Blur,  in  N.  Eisner  &  H.  Zimmermann,  Eds., 
The  Neurosciences  —  From  Basic  Research  to  Therapy:  Proceedings  of  the  29th 
Gottingen  Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag,  Stuttgart,  pp. 
601-602. 

DuBois,  R.A.,  Engel  N.  and  Stuart  G.  (2003)  Design  and  construction  of  a  dynamic 

clamp  device  for  biological  neurons,  in  N.  Eisner  &  H.  Zimmermann,  Eds.,  The 

Neurosciences  —  From  Basic  Research  to  Therapy:  Proceedings  of  the  29th  Gottingen 
Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag,  Stuttgart,  p.  1059. 

DuBois,  R.A.,  Engel  N.  and  Stuart  G.  (2003)  Transient  synchronization  of  cortical 

neurons  using  synthetic  conductance,  in  N.  Eisner  &  H.  Zimmermann,  Eds.,  The 

Neurosciences  —  From  Basic  Research  to  Therapy:  Proceedings  of  the  29th  Gottingen 
Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag,  Stuttgart,  p.  583. 

DuBois,  R.A.  (2003)  Bifurcation  between  quasi-stationary  cortical  activity  states  alters 
the  spatial  and  temporal  distribution  of  response  patterns,  in  N.  Eisner  &  H. 
Zimmermann,  Eds.,  The  Neurosciences  -  From  Basic  Research  to  Therapy:  Proceedings 
of  the  29th  Gottingen  Neurobiology  Conference  (Germany),  Georg  Thieme  Verlag, 
Stuttgart,  p.  582. 


Contract  F49620-01-C-0030  Final  Report 


51 


Harris,  R.A.,  and  O’Carroll,  D.C.  (2002)  Afterimages  in  fly  motion  vision.  Vision 
Research  Vol.  42,  pp.  1701-1714. 

Straw,  A.D.,  O'Carroll,  D.C.,  «&  Shoemaker,  P.A.  (2002)  Adaptive  Motion  Detectors 
Inspired  by  Insect  Vision.  Proceedings  of  the  First  World  Congress  on  Biomimetics  and 
Artificial  Muscles  (Albuquerque,  NM,  December  9-11). 

DuBois,  R.A.  (2001)  Optomotor  reafference  to  the  medulla  and  object  tracking. 
Presented  at  the  International  Conference  for  Invertebrate  Vision  (Backaskog  Castle, 
Sweden,  August  7-12). 


6  INTERACTIONS /TRANSITIONS 


6.1  Participation  /  presentation  at  meetings,  conferences,  and 
seminars 

David  O’ Carroll  and  Patrick  Shoemaker  held  a  collaborator’s  meeting,  including 
O’Carroll  lab  members  R.  DuBois,  T.  Rainsford,  and  A.  Straw,  at  the  University  of 
Adelaide,  21-31  October  2003.  Diuing  this  period.  O’ Carroll  and  Shoemaker  met  with 
COL  Don  Erbschloe,  commanding  officer,  AFOSR,  and  MAJ  Tony  Mitchell,  Technical 
director  at  AOARD,  on  a  site  visit  to  Adelaide. 

Patrick  Shoemaker  attended  the  attended  the  AFOSR  Dynamics  and  Control 
contractors/grantees  meeting  held  9-11  September  2003  in  Destin,  FL. 

David  O’ Carroll  made  a  two- week  visit  in  May- June  2003  to  his  former  laboratory  in 
England  at  the  University  of  Cambridge  (host  Prof  Simon  LaughUn),  and  presented  a 
research  seminar. 

David  O’Carroll  visited  the  University  of  Bonn,  Germany,  in  June  2003  (host  Prof. 
Gerhard  von  der  Emde)  and  presented  a  research  seminar. 

David  O’ Carroll,  Roger  DuBois,  and  Andrew  Straw  attended  the  29*  Gottingen 
Neurobiology  Conference  in  June  2003,  and  presented  a  number  of  posters. 

Roger  DuBois  attended  the  5*  meeting  of  the  German  Neuroscience  Society  in  June 
2003. 

Roger  DuBois  met  with  European  neuroscience  and  biomimetics  researchers, 
including:  Tobi  Delbriick  and  Shih-Chii  Liu  at  the  Institut  fur  Neuroinformatik, 
University  of  Zurich  /  ETH  Zurich;  Prof.  Henry  Markram  at  the  Brain  Mind  Institute  at 
the  Ecole  Polytechnique  Federal  de  Lausanne;  Prof.  Egbert  Welker  at  the  Institut  de 
Biologie  Cellulaire  et  de  Morphologie,  Lausanne;  and  Prof  Greg  Stuart,  University  of 
Freiburg,  Physiology  I,  Freiburg,  Germany,  in  June  2003. 

David  O’ Carroll  and  Patrick  Shoemaker  held  a  collaborator’s  meeting  at  the 
University  of  Adelaide,  4-13  September  2002. 

.  Patrick  Shoemaker  attended  the  AFOSR  Dynamics  and  Control  contractors/grantees 
meeting  held  12-14  August  2002  in  Pasadena,  CA,  and  presented  a  poster  describing 
work  on  this  project. 
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Jordanna  Henry  (graduate  student  in  the  former  O’Carroll  lab  at  University  of 
Washington)  visited  the  O’Carroll  lab  in  April-May  2002  to  analyze  accumulated  data  for 
STMD  responses. 

Patrick  Shoemaker  attended  AFOSR  Workshop  on  Future  Direction  in  Control,  held 
26-27  April  2002  in  Arlington. 

Patrick  Shoemaker  presented  a  seminar  entitled  “Modeling  adaptation  in  insect  visual 
motion  processing”  at  the  University  of  Maryland  on  25  April  2002  (hosted  by  fellow 
AFOSR  grantee  P.S.  Krishnaprasad). 

David  O’ Carroll  and  Patrick  Shoemaker  attended  the  Symposium  on  Microelectronic 
and  Micro-Electro-Mechanical  Systems,  held  17-19  December  2001  in  Adelaide. 
O’Carroll  presented  a  plenary  lecture  entitled  “Biomimetic  visual  detection  based  on 
insect  neurobiology,”  and  Shoemaker  presented  a  poster  entitled  “Implementation  of 
visual  motion  detection  with  contrast  adaptation.” 

Patrick  Shoemaker  attended  the  Conference  on  Cooperative  Control  and  Optimization 
and  AFOSR  Cooperative  Control  Theme  kickoff  meeting  held  12  -  14  November  2001  at 
the  University  of  Florida  in  Gainesville,  FL. 

David  O’ Carroll  and  Jordanna  Henry  attended  the  International  Conference  on 
Invertebrate  Vision,  held  at  Backaskog  Castle,  Sweden,  7-12  August  2001.  Henry 
presented  a  poster  entitled  “Visual  detection  of  prey  by  dragonflies.” 

Roger  DuBois  attended  the  International  Conference  on  Invertebrate  Vision,  held  at 
Backaskog  Castle,  Sweden,  7-12  August  2001,  and  presented  a  poster  entitled 
“Optomotor  reafference  to  the  medulla  and  object  tracking.” 

Patrick  Shoemaker  attended  the  AFOSR  2001  Grantees/Contractors  Meeting  for 
Dynamics  and  Control,  held  30  July  -  2  August  2001  at  Wright-Patterson  AFB,  Dayton, 
OH. 

6.2  Consultative  and  advisory  functions 

From  late  2002  until  the  present,  David  O’Carroll  has  acted  as  a  liason  with  officials 
at  the  Weapons  Systems  Division  (WSD)  of  Australia’s  Defence  Science  and  Technology 
Organisation  (DSTO)  to  investigate  mutual  interests  in  guidance  and  control.  WSD 
supports  biomimetic  work  at  Australian  National  University  (Srinivasan  &  colleagues) 
where  they  presently  fund  a  UAV  project.  As  a  result  of  these  efforts,  (and  relying  on 
materials  provided  with  the  support  of  this  AFOSR  Themes  contract).  Dr.  Farhan  Faruqi 
at  DSTO  has  indicated  the  intention  of  ftinding  a  key  initiative  in  Biomimetics.  DSTO 
intends  to  support  a  full  salary  for  a  Scientist  at  Adelaide  University,  as  well  as  a  similar 
funding  level  for  ANU.  This  will  be  an  initial  3  year  seed  project,  but  with  a  view  to 
establishing  permanent  funding  for  a  centre  of  excellence  (COE)  in  Biomimetics  to  be 
based  in  Adelaide.  Funding  is  currently  on  hold;  however,  this  initiative  will  eventually 
represent  a  substantial  opportunity  to  leverage  funds  supplied  by  the  US  Air  Force. 

6.3  Transitions 

Tanner  Research  (Shoemaker)  in  partnership  with  the  University  of  Adelaide 
(O’Carroll)  in  March  2003  were  awarded  a  Navy  Phase  1  SBIR  project,  entitled  “Insect 
Small  Target  Motion  Detection  for  Seeker  Applications.”  The  contract  number  is 
N00014-03-M-0171,  and  the  COTR,  Dr.  Ron  Scott,  Code  472100D,  NAWCWD,  China 
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Lake,  CA  93555,  phone  (760)  939-4002.  The  proposed  work  relied  heavily  on 
accomplishments  under  this  AFOSR  Themes  contract,  and  the  two  efforts  have  been 
closely  complementary.  During  this  project,  we  examined  in  greater  detail  the  responses 
of  STMD  cells  to  small  moving  targets  against  stereotypic  moving  clutter,  in  the  context 
of  a  realistic  DoD  application  (an  air-to-ground  platform).  However,  the  work  did  not 
transition  to  a  Phase  II  project. 

In  early  2003,  David  O’Carroll  was  successful  in  application  for  a  full  faculty 
position  at  the  University  of  Adelaide.  The  funds  that  had  previously  provided  salary 
support  (under  an  Air  Force  contract  international  research  initiative  (IRI)  contract)  were 
freed  up  and  enabled  the  appointment  of  a  new  Post-Doctoral  Fellow,  Dr.  Tamath 
Rainsford,  with  a  background  in  Physics  and  Neurobiology.  Dr.  Rainsford  has 
contributed  substantially  to  experimental  work  performed  under  this  contract  during 
2003. 


7  Inventions  and  Patent  Disclosures 

A  novel  concept  for  a  log-domain  lowpass  filter  circuit  was  conceived  and  reduced  to 
practice  under  support  of  this  project.  This  circuit  is  capable  of  implementing  very  long 
(biological)  time  constants,  and  in  addition  may  be  operated  in  a  nonlinear  mode  in 
which  it  is  useful  for  the  simulation  of  dynamic  processes  with  fast  onset  and  slow  decay 
(an  important  component  of  small  target-detecting  models).  A  paper  describing  the 
invention  has  been  written  and  is  under  final  review  at  the  journal  Analog  Integrated 
Circuits  and  Signal  Processing.  With  the  statutory  bar  imposed  by  this  publication. 
Tanner  Research  Inc.  expects  to  initiate  the  filing  of  a  patent  on  the  invention  in  the  near 
future.  A  DD  Form  882  is  included  herewith. 

During  development  of  the  stimulus  system  supported  in  part  by  this  project,  Andrew 
Straw  and  David  O’ Carroll  developed  a  novel  motion  animation  technique  that  reduces 
image  ghosting  (a  form  of  temporal  aliasing)  in  apparent  motion  presentations.  The 
method  was  optimized  through  a  model  for  the  physiological  mechanisms  of  temporal 
blur  introduced  by  biological  photoreceptors.  Although  originally  developed  to  remove 
ghosting  artifacts  from  stimulus  sequences  presented  to  insects  during  experimentation, 
the  method  has  widespread  potential  applications  in  real-time  computer  animations  for 
human  observers  (including  video  games).  A  provisional  patent  was  filed  with  the 
Australian  Patent  Office  (provisional  no  2002952815),  and  has  subsequently  lapsed.  We 
are  presently  finalizing  a  manuscript  detailing  the  theory  underlying  the  method  and  are 
considering  seeking  US  patent  protection. 


8  Honors  /  Awards 

The  poster  presented  by  Henry  mentioned  in  Sections  5.1  and  6.1  above  was  runner- 
up  prize-winner  for  the  best  student  presentation  at  the  2001  International  Conference  on 
Invertebrate  Vision. 
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